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Executive Summary 



As the ongoing national indicator of what American students know and can do, the 
National Assessment of Educational Progress (NAEP) in Reading regularly collects 
achievement information on representative samples of students in grades 4, 8, and 12. 
Through “The Nation’s Report Card,” the NAEP Reading Assessment reports how well 
students perform in reading various texts and responding to those texts by answering 
multiple-choice and constructed-response questions. The information NAEP provides 
about student achievement helps the public, educators, and policymakers understand 
strengths and weaknesses in student perfonnance and make infonned decisions about 
education. 

The 2011 NAEP Reading Assessment will measure national, regional, state, and sub- 
group achievement in reading but is not designed to report individual student or school 
perfonnance. The assessment will measure students’ reading comprehension and their 
ability to apply vocabulary knowledge to assist them in comprehending what they read. 
The reading assessment will use the same framework used in 2009. The public will have 
access to performance results and released questions through NAEP reports and websites. 

This document, the Reading Framework for the 2011 National Assessment of Education- 
al Progress, presents the conceptual base for, and discusses the content of, the 
assessment. It is intended for a broad audience. A more detailed technical document, the 
Reading Assessment and Item Specifications for the National Assessment of Educational 
Progress, is available on the web. The specifications will provide information to guide 
passage selection, item development, and other aspects of test development. Both the 
framework and the specifications documents are available to the public at 
http ://www.nagb . org/publications/ frameworks .htm. 

The Governing Board, the policymaking body for NAEP, has stated that the NAEP Read- 
ing Assessment will measure reading comprehension by asking students to read passages 
written in English and to answer questions about what they have read. The framework 
“shall not endorse or advocate a particular pedagogical approach, . . . but shall focus on 
important, measurable indicators of student achievement” (NAGB 2002). Although broad 
implications for instruction may be inferred from the assessment, NAEP does not specify 
how reading should be taught; nor does it prescribe a particular curricular approach to 
teaching reading. 

The NAEP Reading Framework results from the work of many individuals and organiza- 
tions involved in reading and reading education, including researchers, policymakers, 
educators, and other members of the public. Their work was guided by scientifically 
based literacy research that conceptualizes reading as a dynamic cognitive process as 
reflected in the following definition of reading. 
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Reading is an active and complex process that involves: 

• Understanding written text. 

• Developing and interpreting meaning. 

• Using meaning as appropriate to type of text, purpose, and situation. 

This definition applies to the assessment of reading achievement on NAEP and is not 
intended to be an inclusive definition of reading or reading instruction. 

Text Types 



This framework recognizes that reading behaviors such as recognizing and using features 
of text, making sense of sentences and paragraphs, and comprehending vocabulary occur 
regardless of text type. However, other reading behaviors vary with the type of text 
encountered by a reader. Thus, the NAEP Reading Framework includes two types of 
texts on the assessment: literary texts, which include fiction, literary nonfiction, and 
poetry; and infonnational texts, which include exposition, argumentation and persuasive 
text, and procedural text and documents. 

Meaning Vocabulary Assessment 



The NAEP Reading Framework includes a more systematic approach to vocabulary 
assessment than the NAEP Reading Framework used from 1992 through 2007. Vocabu- 
lary assessment will occur in the context of a passage; that is, vocabulary items will 
function both as a measure of passage comprehension and as a test of readers’ specific 
knowledge of the word’s meaning as intended by the passage author. A sufficient number 
of vocabulary items at each grade will provide reliable and valid infonnation about 
students’ vocabulary knowledge. 

Item Design 



The framework includes the following cognitive targets, or behaviors and skills, for items 
from both literary and infonnation texts: locate/recall, integrate/interpret, and critique/ 
evaluate. These cognitive targets illustrate the complex nature of the reading process 
whereas the corresponding behaviors highlight the different behaviors elicited by differ- 
ent text types. To measure these cognitive skills, students will respond to both multiple- 
choice and constructed-response items with varying distributions of question type by 
grade level. Students in grade 4 will spend approximately half of the assessment time 
responding to multiple-choice items and half responding to constructed-response items. 
Students in grades 8 and 12 will spend a greater amount of time on constructed-response 
items. 
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12th Grade NAEP 



In May 2005, the Governing Board adopted a policy statement regarding NAEP and 
12th-grade preparedness. The policy states that NAEP will pursue assessment and report- 
ing on 12th- grade student achievement as it relates to preparedness for postsecondary 
education and training. This policy resulted from recommendations of the Board’s 
National Commission on NAEP 12th Grade Assessment and Reporting in March 2004. 
Subsequent studies and deliberations by the Board took place during 2004 and 2005. 

In reading, the Board adopted minor modifications to the 2009 NAEP Reading Frame- 
work at grade 12 based on a comprehensive analysis of the framework conducted by 
Achieve, Inc. The current version of the reading framework incorporates these modifica- 
tions at grade 12 to enable NAEP to measure and report on preparedness for postsecon- 
dary endeavors. The 2011 NAEP Reading Assessment will use the same framework used 
in 2009. 

Reporting Results 



Results are reported in two ways: as average scores for groups of students on the NAEP 
0-500 scale and as percentages of students who attain each of the three achievement le- 
vels (Basic, Proficient, and Advanced, according to definitions adopted by the Governing 
Board). NAEP scores are always reported at the aggregate level; scores are not produced 
for individual schools or students. 

Reporting Trend Data 



The Reading Framework for the 2009 National Assessment of Educational Progress 
replaced the framework first used for the 1992 reading assessment and then for subse- 
quent reading assessments through 2007. Compared to the previous framework, the 2009 
reading framework included more emphasis on literary and informational texts, a redefi- 
nition of reading cognitive processes, a new systematic assessment of vocabulary know- 
ledge, and the addition of poetry to grade 4. 

The 2009 NAEP Reading Report Card included trend data on student reading perfor- 
mance from 1992 to 2009. Results from special analyses determined the 2009 reading 
assessment results could be compared with those from earlier assessment years. These 
special analyses started in 2007 and included in-depth comparisons of the frameworks 
and the test questions, as well as an examination of how the same students perfonned on 
the 2009 assessment and the earlier assessment. A summary of these special analyses and 
an overview of the differences between the previous framework and the 2009 framework 
are available on the web at http://nces.ed.gov/nationsreportcard/reading/ trend_study.asp. 
The 2011 NAEP Reading Report Card will report trends in student reading performance 
from 1992 to 2011. 
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Sample Items 



Sample passages and items released to the public may be viewed on the NAEP Questions 
Tool at http://nces.ed.gov/nationsreportcard/itmrlsx/. The Questions Tool also includes 
perfonnance results, scoring rubrics, and student responses to NAEP reading items. 
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Preface by the 
National Assessment Governing Board 



In a modern society, the ability to read well is the cornerstone of a child’s education. In a modern 
economy, literacy is a prerequisite for a successful life. 

In their early years of schooling, children learn to draw meaning and pleasure from the words on 
a page, which gives them a sense of accomplishment. Throughout the remainder of their school- 
ing, reading is the critical skill they use for learning in all parts of the curriculum. For adults, 
reading is a key means to learn and do our jobs; it is also a source of enjoyment and an essential 
way we connect with family, friends, and the world around us. The ability to read critically and 
analytically is crucial for effective participation in America’s democratic society. 

The Reading Framework for the 2011 National Assessment of Educational Progress sets forth 
the design of a test of reading comprehension. The exam requires students to read passages of 
written English text — either literary or infonnational — and to answer questions about what they 
have read. In some cases, the questions deal with facts in the text or vocabulary. In other cases, a 
complete answer requires a clear analysis or coherent argument supported by sound evidence 
from the text. 

This is the second reading framework approved by the National Assessment Governing Board. It 
replaces the framework that was used in the National Assessment of Education Progress (NAEP) 
from 1992 to 2007. This new reading framework is the result of extraordinary effort and com- 
mitment by hundreds of people across the country, including some of the nation’s leading figures 
in reading research, assessment, and instruction. 

The new framework incorporates the following key features: 

• Its design is based on current scientific research in reading. In keeping with Governing 
Board policy, it does not advocate a particular approach to instruction, but rather focuses 
on important, measurable indicators of student achievement. 

• The framework’s content and preliminary achievement standards at grade 12 embody 
reading and analytical skills the project committees believe are needed for rigorous col- 
lege-level courses and other productive postsecondary endeavors. 

• In preparing the framework, extensive use was made of international reading assessments 
and exemplary state standards. 

• Vocabulary is measured explicitly. Word meanings will be tested in context, and enough 
vocabulary items will be included to report useful infonnation on the extent of vocabu- 
lary knowledge. 
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• Poetry is assessed in grade 4 as well as in grades 8 and 12. Previously, NAEP assessed 
poetry in grades 8 and 12 only. Poetry is a form of text that is rich in meaning and 
involves a high level of abstraction in language and ideas. 

• Multiple-choice and constructed-response items (both short and extended) are included at 
all grades. In grades 8 and 12, students will be expected to spend about 60 percent of 
assessment time on constructed-response questions; at grade 4, about 50 percent. 

• Descriptions of reading material to be used in the assessment and target skills to be tested 
are delineated in a series of charts that provide clear guidance to those developing the 
assessment and clear information to the public. 

• Achievement will be reported on an overall cross-grade scale, allowing NAEP to show 
the development of reading skills throughout years of schooling as well as the wide varia- 
tions in particular grades. Clear standards for grade-level expectations will be established. 

• Separate subscales will be reported for literary and infonnational text, as has been done 
on international reading assessments. 

The Governing Board would like to thank the hundreds of individuals and organizations whose 
time and talents contributed to this reading framework. The framework process was conducted 
through a contract with American Institutes for Research (AIR). Both AIR and another organiza- 
tion, the Education Leaders Council, prepared literature reviews and issues papers, which pro- 
vided different perspectives and served as the basis for extensive discussions by the Reading 
Framework Steering and Planning Committees. These committees, working over a period of 14 
months, included teachers, reading researchers, local and state policymakers, testing experts, and 
business and public representatives. Many of these individuals have played important roles in 
other major projects, including the National Reading Panel, international reading assessments, 
the RAND Reading Study Group, and the American Diploma Project. 

In addition, the Board convened an independent external review panel comprised of eminent 
reading scholars, authors, and curriculum specialists. Their charge was to conduct an in-depth 
analysis of the framework draft, including its research base and design. These individuals played 
an important role in shaping the framework adopted by the Board. The Board also received wide 
comments on the draft framework through Internet reviews, a public forum held in Washington, 
D.C., and numerous meetings with state and local educators and policymakers across the 
country. 

We believe the framework will provide a rich and accurate measure of the reading comprehen- 
sion and analytical skills that students need both for their schooling and for their lives. Develop- 
ment of these reading skills is the responsibility of all teachers — not only English teachers but 
also teachers across the curriculum — and also involves the expectations of parents and society. 
The Board hopes that this reading framework will serve not only as a significant national meas- 
ure of how well students read, but also as a catalyst to improve reading achievement for the ben- 
efit of students themselves and for our nation. 
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Chapter One 



Overview 

Since 1969, the National Assessment of Educational Progress (NAEP) has been an ongo- 
ing national indicator of what American students know and can do in major academic 
subjects, including reading in English. NAEP reading assessments have been adminis- 
tered on a regular schedule to students in grades 4, 8, and 12. Under the No Child Left 
Behind Act of 2001 (NCLB), NAEP will assess reading in grades 4 and 8 every 2 years. 
NAEP will also measure reading in grade 12 every 4 years. 

The Reading Framework for the 2011 National Assessment of Educational Progress is 
one of two documents that describe the assessment; it is intended for a general audience 
and presents the conceptual base and content of the assessment. The second document is 
the Reading Assessment and Item Specifications for the National Assessment of 
Educational Progress and is intended for a more technical audience, including the 
National Center for Education Statistics (NCES) and the contractors that will develop the 
NAEP Reading Assessment. The specifications provide the “test blueprint”; that is, 
information about passage selection, item development, and other aspects of test 
development. 

NAEP Overview 



The National Assessment Governing Board — the policymaking body for NAEP — has 
defined several parameters for the reading assessment. First, the NAEP assessment will 
measure reading comprehension in English. On the assessment, students will be asked to 
read passages written in English and to answer questions about what they have read. 
Second, because this is an assessment of reading comprehension and not listening com- 
prehension, NAEP does not allow passages to be read aloud to students as a test accom- 
modation. Third, under Board policy, the framework “shall not endorse or advocate a 
particular pedagogical approach, . . . but shall focus on important, measurable indicators 
of student achievement” (NAGB 2002). Although broad implications for instruction may 
be inferred from the assessment, NAEP does not specify how reading should be taught; 
nor does it prescribe a particular curricular approach to teaching reading. 

Reading passages are selected to be interesting to students nationwide, to represent high- 
quality literary and informational material, and to be free from bias. Students respond to 
both multiple-choice and constructed-response items. In total, the NAEP assessments at 
grades 4, 8, and 12 are extensive enough to ensure that results can be reported validly, but 
no single student participates in the entire assessment. Instead, each student reads 
approximately two passages and responds to questions about what he or she has read. 
NAEP assessments are administered to random samples of students designed to be 
representative of the nation, different regions of the country, states, and large urban 
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districts. As discussed in chapter three, NAEP results are reported for groups of students; 
no data are reported for individual students. Since 1992, states have been able to obtain 
state-level data on students’ reading achievement. Since 2003, a number of large urban 
school districts were able to obtain data about their students’ reading achievement. 
Results are reported in documents such as the NAEP Reading Report Card issued 
following each administration of the reading assessment; through special, focused 
reports; and through electronic means. 

Data are also collected that allow comparison of students’ reading achievement over 
extended periods of time in a separate Long-Term Trend NAEP. These assessments, giv- 
en at the national level only, have been administered in the same form since 1971 and 
provide the only available measure of extended long-tenn trends in reading achievement. 

Purpose of NAEP Under NCLB Legislation 

The NAEP Reading Framework is consistent with NCLB legislation. The NAEP legisla- 
tion, as amended under NCLB and the later National Assessment of Educational Progress 
Reauthorization Act (NAEPRA) of 2002, specifies that NAEP’s purpose is “to provide, 
in a timely manner, a fair and accurate measurement of student academic achievement 
and reporting of trends in such achievement in reading, mathematics, and other sub- 
jects[s] . . .” (section 303(b)(1), National Assessment of Educational Progress Reauthori- 
zation Act, P.L. 107-279). The NAEP reading data will measure national, 
regional, and subgroup trends in reading achievement but will not target the perfonnance 
of individual students or schools. In further accordance with NCLB, the NAEP Reading 
Assessment will be administered every 2 years at grades 4 and 8 and the resulting data 
will be widely reported in a timely fashion. Finally, NAEPRA specifies that although the 
public will have full access to NAEP results and released test questions, NAEP will not 
seek to influence the curriculum or assessments of any state. 

Definition of Reading for NAEP 

The NAEP Reading Assessment is guided by a definition of reading that reflects 
scientific research, draws on multiple sources, and conceptualizes reading as a dynamic 
cognitive process. This definition applies to the assessment of reading achievement on 
NAEP and states that reading is an active and complex process that involves: 

• Understanding written text. 

• Developing and interpreting meaning. 

• Using meaning as appropriate to type of text, purpose, and situation. 

Terms used in the definition can be further explained as follows: 

Understanding written text: Readers attend to ideas and content in a text by locating 
and recalling information and by making inferences needed for literal comprehension of 



NAEP 2011 Reading Framework 



2 




the text. In doing so, readers draw on their fundamental skills for decoding printed words 
and accessing their vocabulary knowledge. 

Developing and interpreting meaning: Readers integrate the sense they have made of 
the text with their knowledge of other texts and with their outside experience. They use 
increasingly complex inferencing skills to comprehend infonnation implied by a text. As 
appropriate, readers revise their sense of the text as they encounter additional information 
or ideas. 

Using meaning: Readers draw on the ideas and infonnation they have acquired from text 
to meet a particular purpose or situational need. The use of text may be as straightforward 
as knowing the time when a train will leave a particular station, or it may involve more 
complex behaviors such as analyzing how an author developed a character’s motivation 
or evaluating the quality of evidence presented in an argument. 

Text: As used in the assessment, the term reflects the breadth of components in typical 
reading materials. Thus, text on the assessment will include literary and infonnational 
passages and may contain noncontinuous print material such as charts. Texts selected for 
inclusion on the assessment represent practical, academic, and other contexts and are 
drawn from grade-appropriate sources spanning the content areas. 

Purpose: Students’ purpose for reading the passages presented on NAEP is determined 
by the assessment context; thus, the influence of purpose on readers’ comprehension is 
somewhat limited. 

Situation: The situation for reading often detennines the way that readers prepare for and 
approach their task. They consider why they are reading (e.g., to study, to relax), how 
much they know about the topic, and other concerns that shape the time they will spend 
reading. 

Factors That Influence Reading Performance 

Factors related to the text being read and to readers’ backgrounds and experiences influ- 
ence reading performance. For example, understanding the vocabulary, concepts, and 
structural elements of the text contributes to readers’ successful comprehension. Com- 
prehension is also affected by readers’ background knowledge and by the context of the 
reading experience. The background knowledge that students bring to the NAEP Reading 
Assessment differs widely. To accommodate these differences, passages will span diverse 
areas and topics and will be as engaging as possible to the full range of students in the 
grades assessed. 

The purpose for reading also influences performance. In the case of the NAEP Reading 
Assessment, purpose is detennined by the assessment context; thus, the influence of pur- 
pose on readers’ comprehension is somewhat limited. For this reason, the definition of 
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reading presented earlier should be considered as a guide for the NAEP Reading Assess- 
ment, not as an inclusive definition of reading. The definition pertains to how NAEP 
defines reading for the purpose of this assessment. It does not address the issue of how 
students should be taught to read. 

Text comprehension is influenced by readers’ ability to apply the essential components of 
reading: phonemic awareness, phonics knowledge, fluency, and understanding of word 
meanings or vocabulary. Without these foundational skills, comprehension will not 
occur. By grade 4, when the NAEP Reading Assessment is first administered, students 
should have a well-developed understanding of how sounds are represented alphabetical- 
ly and should have had sufficient practice in reading to achieve fluency with different 
kinds of texts (National Research Council 1998). Because NAEP tests at grades 4, 8, and 
12, the assessment focuses on students’ reading comprehension, not their foundational 
skills related to alphabetic knowledge. 1 

As discussed further in chapter two, the association between vocabulary knowledge and 
comprehension is strong; students who know the meanings of many words and who also 
can use the context of what they read to figure out the meanings of unfamiliar words are 
better comprehenders than those who lack these attributes (National Reading Panel 
2000a). In the NAEP Reading Assessment, vocabulary will be assessed systematically 
through carefully developed items that measure students’ ability to derive the meanings 
of words within the context of the passages they read. 

Nature of Reading Behaviors 

Reading is an active and complex process that involves multiple different behaviors. 
Readers often begin by forming an overview of text and then search for information to 
which they must pay particular attention. Following this initial overview, readers 
progress with different levels of interaction with text, including interpreting and evaluat- 
ing what they read. By drawing on previous reading experiences and prior knowledge, 
they form hypotheses about what the text will communicate and revise their initial ideas 
and their knowledge base as their reading continues. Readers continuously acquire new 
understandings and integrate these into their ongoing process of building comprehension. 
Good readers monitor their understanding of text, recognize when text is not making 
sense, and employ a range of strategies to enhance their comprehension. Good readers 
also evaluate the qualities of text, and these evaluations can affect whether a text is 
remembered or has an impact on readers’ knowledge, attitudes, or behaviors (Pressley 
and Afflerbach 1995; Ruddell and Unrau 1994). Depending on the situation and purpose 
for reading, good readers can use the ideas and information they acquire from text to, for 
example, expand their thinking about a topic, perfonn a specific task, or draw conclu- 
sions or make generalizations about what they have read. 



'NAEP investigated the relationship between oral fluency and reading comprehension in two special 
studies, in 1992 and 2002. 
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Definitions of Reading Tha t Ha ve Informed Framework Development 

The definition of reading for the NAEP Reading Assessment is derived from several 
sources and grounded in scientific research on reading. Among the sources are NCLB, 
several important research reports on reading, and the definitions of reading that guide 
the development of international reading tests. Each source has contributed important 
ideas to the definition of reading used for the NAEP Reading Assessment. 

NCLB posits that reading has five essential components: phonemic awareness, know- 
ledge of phonics, reading fluency, vocabulary, and comprehension. The NAEP Reading 
Assessment measures students’ meaning vocabulary and comprehension. To demonstrate 
comprehension of what they read, students use their phonemic awareness and knowledge 
of phonics. Their ability to read the passages and test questions with minimal effort 
reflects their fluency. Students draw on their vocabulary knowledge throughout the 
assessment and specific items ask about carefully selected target words in each reading 
passage. 

The National Reading Panel (NRP) (National Institute of Child Health and Human 
Development 2000), a congressionally mandated commission, conducted an extensive, 
evidence-based study of research literature on reading acquisition, reading growth, and 
other relevant topics. The NRP report was an important foundation for NCLB, highlight- 
ing the importance of alphabetics (phonemic awareness and phonics), fluency, vocabu- 
lary, and comprehension. 

Three important definitions of reading influenced the development of the definition of 
reading for the NAEP Reading Assessment. The first comes from Reading for Under- 
standing: Toward an R&D Program in Reading Comprehension (RAND Reading Study 
Group 2002), frequently referred to as the RAND Report. This report was prepared by the 
RAND Reading Study Group under the auspices of the Office of Educational Research 
and Improvement of the U.S. Department of Education. Guiding the work of the study 
group was the following definition of reading: 

Reading comprehension [is] the process of simultaneously extracting and 
constructing meaning through interaction and involvement with written 
language. It consists of three elements: the reader, the text, and the activity 
or purpose for reading (p. 11). 

The second important definition was the foundation for item development for the 
Progress in International Reading Literacy Study (PIRLS) (Campbell et al. 2001). PIRLS 
was first administered to 9-year-old students in 35 countries in 2001. PIRLS defines read- 
ing literacy as: 

The ability to understand and use those written forms required by society 
and/or valued by the individual. Young readers can construct meaning 
from a variety of texts. They read to learn, to participate in communities of 
readers, and for enjoyment (p. 3). 
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The Programme for International Student Assessment (PISA) (OECD 2000) represents an 
international collaborative effort to assess what 15 -year-old students know and can do in 
reading, mathematics, and science. PISA defines reading literacy as: 

Understanding, using, and reflecting on written texts, in order to achieve 
one’s goals, to develop one’s knowledge and potential, and to participate 
in society (p. 18). 

The RAND Report, PIRLS, and PISA offer support to the definition for reading advocated 
in the NAEP Reading Framework. All three stress that reading is an active, complex, and 
multidimensional process undertaken for many different purposes. 

Overview of NAEP Reading Assessment 



This reading assessment will use the same framework used in 2009. The NAEP Reading 
Assessment will include two distinct types of text at grades 4, 8, and 12. Doing so will 
allow the development of items that measure students’ comprehension of the different 
kinds of text they encounter in their school and out-of-school reading experiences. The 
reasons for including literary and informational text are presented next, followed by 
explanations of the characteristics of each text type included on the assessment. The 
NAEP Reading Assessment will also include items that measure students’ ability to apply 
their knowledge of vocabulary as an aid in their comprehension process. 

Neither computer-based electronic text nor drama will be included on the NAEP Reading 
Assessment. NAEP is committed to presenting authentic texts as stimulus material on its 
reading assessments, and it is difficult to include these kinds of text in ways that reflect 
how students actually read them in and out of school. The paper-and-pencil fonnat most 
commonly used in NAEP reading assessments precludes students’ navigating through 
different components of text as they do when they read electronic text. Furthermore, 
dramatic selections are usually too long to fit within the word-length parameters for 
passages included on the assessment. 

NAEP assesses reading skills that students use in all subject areas and in their out-of- 
school and recreational reading. By design, many NAEP passages require interpretive 
and critical skills usually taught as part of the English curriculum. However, NAEP is an 
assessment of varied reading skills, not a comprehensive assessment of literary study. 

The development of the broad range of skills that the nation’s students need to read suc- 
cessfully in both literary and infonnational texts is the responsibility of teachers across 
the curriculum, as well as of parents and the community. 

Commonalities in Reading Beha viors Across Text Types 

The framework recognizes that even though there are substantial differences in reading 
behaviors for different text types, there are also great similarities. Regardless of the type 
of text, the reader must access the words in the text, recognize and use the structure of the 
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text, make sense of sentences and paragraphs, and comprehend what has been read. 
Equally, vocabulary is a critical element in comprehending any kind of text. 

Text Characteristics: Literary and Informational Texts 

Research on the nature of text and on reading processes has suggested that the characte- 
ristics of literary and infonnational text differ dramatically. For the most part, the re- 
search literature suggests that readers pay attention to different aspects of text as they 
seek to comprehend different text types (Pearson and Camperell 1994; Pressley 2000; 
Purves 1973). Additionally, the PIRLS report shows that students in the United States 
scored higher on the Literary Subscale (at 550) than on the Infonnational Subscale (at 
533), further substantiating the difference in the strategies needed for the two text types 
(OECD 2000). An earlier international study reported that patterns of student responses 
to literature were influenced by the nature of the selections they were given to read. Dif- 
ferent 

literary samples elicited different responses from students with some consistency across 
cultures and school systems (Purves 1973). Drawing on this extensive research base, the 
NAEP Reading Framework includes two major types of text: literary and infonnational. 
Well-crafted nonfiction work with strong literary characteristics will be classified as 
literary text and documents such as tables, graphs, or charts will be included in the 
informational category. 

Literary and informational texts for the NAEP Reading Assessment are separated for two 
primary reasons: the structural differences that mark the text types and the purposes for 
which students read different texts. Exhibits 3 and 4 in chapter two present details about 
the kinds of literary and informational texts to be included on the NAEP Reading Asess- 
ment and about the features of these texts for which items will be written. 

Structural Differences in Text 

Literary and informational texts are marked by distinct structural characteristics that 
readers rely on as they seek to understand what they read (Goldman and Rakestraw 
2000). For example, research on literary text (Graesser, Golding, and Long 1991) has 
pointed out that stories and novels are characterized by a coherent text structure known as 
“story grammars.” Research on infonnational or expository text (Kobayashi 2002) has 
indicated that such texts possess distinct organizational patterns, such as sequence or 
comparison and contrast, designed to help readers organize their emerging sense of what 
the text is communicating. These structures are distinct from story grammars. The nature 
of texts affects comprehension, and different text types must be read in different ways 
(Pearson and Camperell 1994). Good readers adjust their reading behaviors to accommo- 
date the kinds of text they are reading. 
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Purposes for Reading 



A second reason for separating text types is that readers often read literary and informa- 
tional texts for different purposes. The definition of reading that guides the NAEP 
Reading Assessment specifically states that readers read for different purposes, which are 
often reflected in their selection of literary or informational texts. The purpose set for 
reading a text often determines how a student reads that text. Literary texts, such as 
stories, drama, essays, or poetry, are frequently read for pleasure or for new perspectives 
on time, place, human nature, or feelings; they are often read from beginning to end. The 
ultimate utility of informational text is detennined by how well it conveys information or 
ideas. These differences in reading purpose are, of course, penneable. For example, well- 
crafted informational text is often read for appreciation and enjoyment, in addition to the 
infonnation that the text can provide. 

Features That Distinguish Text Types 

Several features distinguish literary and informational texts. Skilled writers understand 
that different kinds of text need different structural patterns, and good readers are able to 
use these specific text features as aids in comprehension. 

Literary Texts 

The NAEP Reading Assessment will present reading passages (i.e., stimulus material) 
drawn from three categories of literary text: 

• Fiction. 

• Literary nonfiction, such as essays, speeches, and autobiographies or biographies. 

• Poetry. 

The structural patterns of fiction (i.e., short stories and novels) have been studied exten- 
sively. Although many researchers have suggested different ways to name the elements 
of a story (Stein and Glenn 1979), there is general agreement that a story consists of the 
following components: the setting or settings; a simple or complex plot consisting of a 
series of episodes and delineating a problem to be solved; the problem or conflict, which 
requires characters to change, revise plans, or face challenges as they move toward reso- 
lution; and a reaction that expresses the protagonist’s feelings about his or her goal 
attainment or relates to the broader consequences of the conclusion of the story. This 
structure is often referred to as a story grammar. Characters populate each story, in major 
or minor roles; themes or major ideas are stated either implicitly or explicitly. 

Works of literary nonfiction such as biographies, essays, and speeches employ distinct, 
varied structural patterns and literary features to reflect their purpose and audience. These 
works may not only present information and ideas but also employ distinctly literary 
elements and devices to communicate their message and to make their content more 
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accessible to readers. Biographies and autobiographies, for example, usually follow a 
structure that in many ways mirrors the story structure of fictional works, and they may 
employ literary devices, but they also present infonnation. Literary essays and speeches 
may be structured differently but also draw on literary devices. The Gettysburg Address, 
for example, might be viewed simply as an argumentative text or as a dedication or a eu- 
logy, but it is more appropriately viewed as a sophisticated literary text. Readers ap- 
proach texts of this type not only to gain enjoyment and information, but also to leam and 
to appreciate the specific craft behind authors’ choices of words, phrases, and structural 
elements. 

Like fiction and literary nonfiction, poetry demonstrates specific text characteristics, but 
these characteristics are different from those found in continuous prose (Hanauer forth- 
coming). Some poetry possesses very rhythmic or metrical patterns, and some is written 
as “free verse” without a regular line pattern. Poetry is a highly imaginative fonn of 
communication in that poets try to compress their thoughts in fewer words than would be 
used in ordinary discourse or in prose (Frye 1964). Because the language is often brief 
and concise, poems employ picturesque and evocative words as well as similes, meta- 
phors, personification, imagery, and other devices that convey the symbolic nature of the 
ideas, emotions, and actions being expressed. Poetry often involves a high level of 
abstraction in language and ideas, and requires specific critical thinking skills not found 
in other types of literary works. For these reasons, it is important that NAEP include 
poetry on the assessment at grades 4, 8, and 12. 

Informational Texts 

For the NAEP Reading Assessment, infonnational texts will be classified into three broad 
categories: 

• Exposition. 

• Argumentation and persuasive text. 

• Procedural text and documents. 

Infonnational text, specifically exposition, argumentation, and persuasive text, does not 
have a single, identifiable structure. Rather, different types of informational text exhibit 
distinct structural features. The most common structural patterns for continuous exposito- 
ry, argumentative, and persuasive text can be summarized as follows (Bovair and Kieras 
1991; Meyer 1975; Goldman and Rakestraw 2000; Kobayashi 2002): 

Description: A descriptive text structure presents a topic with attributes, specifics, or set- 
ting information that describes that topic. 

Sequence: Ideas are grouped on the basis of order or time. 

Causation: The text presents causal or cause-and-effect relationships between the ideas 
presented in the text. 
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Problem/Solution: The main ideas are organized into two parts: a problem and a subse- 
quent solution that responds to the problem, or a question and an answer that responds to 
the question. 

Comparison: Ideas are related to one another on the basis of similarities and differences. 
The text presents ideas organized to compare, to contrast, or to provide an alternative 
perspective. 

Expository text, argumentation, and persuasive text often contain pictures, charts, tables, 
and other graphic elements that augment text and contribute to its meaning. Ancillary 
aids such as headings, bolded text, or bulleted lists emphasize specific components of the 
text to reinforce authors’ messages. Literary texts differ in that illustrations, pictures, or 
other nonprint elements (when present) may aid readers in understanding the text but are 
not usually critical for comprehension. 

The first kind of infonnational text on the NAEP Reading Assessment, exposition , 
presents infonnation, provides explanations and definitions, and compares and contrasts. 
Textbooks, news stories, and infonnational trade books are examples of expository text. 
Texts classified as argumentation or persuasive text accomplish many of these same goals 
but can be distinguished by their particular purpose and by the features that authors select 
to accomplish their goals for writing. 

The second category of infonnational text includes argumentation and persuasive text 
(Driver, Newton, and Osborne 2000; Osborne 2002; Wineburg 1991). Argumentation 
seeks to influence through appeals that direct readers to specific goals or try to win them 
to specific beliefs. Authors of persuasive writing must establish their credibility and au- 
thority if their writing is to be successful. Examples of persuasive text are political 
speeches, editorials, and advertisements. 

The third type of infonnational text is often categorized as procedural text or documents 
(Kirsch and Mosenthal 1990; Mosenthal 1996; Mosenthal 1998). Procedural texts convey 
information in the fonn of directions for accomplishing a task. A distinguishing characte- 
ristic of such text is that it is composed of discrete steps to be perfonned in a strict 
sequence with an implicit end product or goal. After reading the text, the reader should be 
able to reach a goal or complete a product. Examples include (but are not limited to) ma- 
nuals and product support materials, directions for art activities and hobbies, and so on. 
Procedural texts may include information arranged in graphs, charts, or maps, in addition 
to prose. 

Document texts in a variety of forms will also be represented on the NAEP Reading 
Assessment. Documents include graphical representations, often as multimedia elements 
that require readers to draw on infonnation presented as short continuous prose and also 
as columns, matrices, or other formats. Document structures can be simple or complex, 
and can present information in a straightforward way as in a simple list or pie graph with 
clearly delineated elements or embed or “nest” infonnation within a document’s struc- 
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ture. Documents are used frequently in schools and in society. Textbooks often include 
graphs, tables, and illustrations to accompany and expand on traditional text. Forms are 
also common (such as applications), as are procedural texts (such as manuals and direc- 
tions). Documents have implicit procedures embedded within them. Often, readers must 
“cycle” through the document or the set of procedures to gain needed information or 

to answer specific questions. For example, instructions suggest the manner in which an 
application is to be completed. 

Informational text will be included at all levels of the NAEP Reading Assessment. 
Documents embedded in text will be used at grades 4 and 8; stand-alone documents that 
provide enough information to support item development may be used at grade 12. 
Chapter two describes the criteria for evaluating examples and noncontinuous text and 
documents for inclusion. 

Percentage of Passages by Text Type and Grade 

Exhibit 1 shows the distribution of literary and informational passages on the assessment. 
The percentage listed for literary texts encompasses all three categories of text: fiction, 
literary nonfiction, and poetry. The percentage for informational text likewise includes 
exposition, argumentation and persuasive texts, and procedural texts and documents. 

The Specifications for the NAEP Reading Assessment will detail how these percentages 
are to be distributed across grades 4, 8, and 12. 

The distribution reflects the kinds of texts that students read across the curriculum as they 
progress through elementary, middle, and high school (Alexander and Jetton 2000). It 
further reflects the distribution of text types on many state reading tests designed to 
reflect what students read across the curriculum. 



Exhibit 1. Percentage distribution of literary and informational passages 



Grade 


Literary 


Informational 


4 


50 


50 


8 


45 


55 


12 


30 


70 



Mixed Texts 

Many of the texts that convey information have been termed mixed texts (Alexander and 
Jetton 2000). This type of text is common in classroom reading as students are introduced 
to informational texts as a genre distinct from the “stories” common in lower grades 
(Duke 2000; Leu and Kinzer 2000). Examples include historical or scientific accounts 
presented in quasi-narrative form but used to communicate information. Their literary 
qualities (for example, literary elements and devices) will determine their classification as 
literary or informational. 
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Multiple Texts 



A common task for readers at all grades is integrating information across a set of texts. It 
is often the case that readers have multiple questions for which they need or want 
answers. A single text may answer some questions incompletely, or a single text might 
contain answers for only a portion of the questions a reader has. The solution is to use 
other texts to find additional information. In consulting multiple texts, readers must 
engage in all the processes to read individual texts, and they must also engage in other 
processes to compare those texts on multiple dimensions and decide on their accuracy, 
bias, and credibility. These skills need to be assessed to see how well students can read 
and comprehend texts that contain different information, reach different conclusions 
about the same material, or have different levels of credibility. Continuing the use of 
intertextual passage sets as part of the NAEP Reading Assessment approximates the 
authentic task of reading and comparing multiple texts. 

Vocabulary Assessment on the NAEP Reading Assessment 

The Governing Board has endorsed the idea of measuring students’ vocabulary as part of 
the reading assessment and supports an approach that assesses vocabulary in the context 
of the reading passages. The goal of vocabulary assessment will be to measure students’ 
meaning vocabulary, which can be defined as follows: 

Meaning vocabulary is the application of one’s understanding of word meanings 

to passage comprehension. 

The proposed method of assessing meaning vocabulary on the NAEP Reading Assess- 
ment assumes that the ability to gain a sense of the meaning of all or most words in a 
passage, especially those words that convey important information linked to central ideas 
of the passage, is a necessary condition for comprehension. NAEP meaning vocabulary 
items will target words already present in the NAEP reading comprehension passages. 
Candidate words must convey important meaning linked to the central idea(s) of the 
passage; comprehension would likely be disrupted if the meaning of the test word was 
not known. It is anticipated that each passage will have approximately two vocabulary 
items. The vocabulary assessment is explained in detail in chapter two. 

Assessing Students With Special Needs 

The NAEP Reading Assessment is designed to measure the academic achievement of all 
test takers at a given grade level; hence, students with disabilities and English language 
learners are included in the assessment sample. The assessment is administered to Eng- 
lish language learners and students with disabilities who, based on inclusion criteria 
provided by NAEP, are capable of participating. Special care is taken in designing and 
developing the assessment to ensure that these students, along with all others, find the 
passages and items accessible. For example, passages that might require specific back- 
ground or experiential knowledge for comprehension are not included in the assessment. 
Items are written in plain language without jargon or complex syntactical structures. 
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Some students may need accommodations to be able to participate in the NAEP Reading 
Assessment. NAEP attempts to provide accommodations to students that match the way 
in which they are tested in school as long as those accommodations do not alter the con- 
struct being measured. For example, large-print versions are made available for students 
with visual impairments; students with disabilities may be given one-on-one or small- 
group testing situations or extended time to complete the assessment. Some students, for 
example those who are learning English, may have the test directions (but not the 
passages or items) read orally to them. Other students may benefit from having a trained 
aide transcribe dictated responses for them. Accommodations may be provided in combi- 
nation, for example, extended testing time and individual administration of the assess- 
ment. 

Comparison of 1992-2007 NAEP Reading Framework and 2009-2011 NAEP 
Reading Framework 

The framework for the 2009-2011 NAEP Reading Assessment replaces a framework 
developed for the 1992 assessment. The previous framework was refined during its use to 
reflect more clearly the goal of precisely measuring students’ reading skills and strategies 
and was reissued for the 2003 assessment. The new framework honors many aspects of 
the previous one, but also introduces some changes that can lead to better measurement 
and more precise reporting of assessment results. Important changes featured in the new 
NAEP Reading Framework follow: 

• An assessment design based on current scientific reading research. 

• Use of international reading assessments to inform the NAEP framework. 

• More focused measurement of vocabulary. 

• Measurement of reading behaviors (cognitive targets) in a more objective manner. 

• Distinction of cognitive targets relevant to literary and infonnational text. 

• Use of expert judgment, augmented by readability formulas, for passage selection. 

• Testing of poetry at grade 4 in addition to grades 8 and 12. 

• Special study of vocabulary to inform development of the assessment. 

Key similarities and differences between the two frameworks are presented in exhibit 2. 
Chapter two explains the proposed content and design of the assessment. The content and 
cognitive targets, as operationalized to reflect the definition of reading presented earlier 
in chapter one, will yield passages and items that reflect the complex interaction of the 
reader, the text, and the context of the assessment. 
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Exhibit 2. Similarities and differences: 1992-2007 and 2009-2011 NAEP reading 

frameworks 





1992-1997 NAEP 
Reading Framework 


2009-2011 NAEP 
Reading Framework 


Content 


Content of 
assessment: 

• Literary 

• Informational 

• Document 


Contexts for reading: 

• For literary 
experience 

• For information 

• To perform task 


• Literary text 

• Fiction 

• Literary nonfiction 

• Poetry 


• Informational text 

• Exposition 

• Argumentation and 
persuasive text 

• Procedural text and 
documents 




Stances/aspects of reading: 


Cognitive targets distinguished by text type 


Cognitive 

Processes 


• Forming general understanding. 

• Developing interpretation. 

• Making reader/text connections. 

• Examining content and structure. 


Locate/ 

recall 


Integrate/ 

interpret 


Critique/ 

evaluate 


Vocabulary 


Vocabulary as a “target” of item 
development, with no information reported 
on students’ use of vocabulary knowledge 
in comprehending what they read. 


Systematic approach to vocabulary assessment 
with potential for a vocabulary subscore. 


Poetry 


Poetry included as stimulus material at 
grades 8 and 12. 


Poetry included as stimulus material at all grades. 


Passage 

Source 


Use of intact, authentic stimulus material. 


Use of authentic stimulus material plus some 
flexibility in excerpting stimulus material. 


Passage 

Length 


Grade 4: 250-800 
Grade 8: 400-1,000 
Grade 12: 500-1,500 


Grade 4: 200- 
Grade 8: 400- 
Grade 12: 500 


800 

1,000 

-1,500 






Passage 

Selection 


Expert judgment as criterion for passage 
selection. 


Expert judgment and use of at least two research- 
based readability formulas for passage selection. 


Item Type 


Multiple-choice and constructed-response 
items included at all grades. 


Multiple-choice and constructed-response items 
included at all grades. 
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Chapter Two 



Content and Design of NAEP in Reading 

This chapter presents the content and design of the NAEP Reading Assessment. Key 
sections of the chapter are as follow: 

• Texts to be included on the NAEP Reading Assessment 

• Characteristics of texts selected for inclusion on the NAEP Reading Assessment 

• Literary text 

• Infonnational text 

• Vocabulary on the NAEP Reading Assessment 

• Cognitive targets for the NAEP Reading Assessment 

• Item types on the NAEP Reading Assessment 

Texts on the NAEP Reading Assessment to be Included 



The National Assessment of Educational Progress in Reading will assess students’ 
comprehension of literary and infonnational passages. Within these passages, vocabulary 
will also be assessed. Chapter one presented the rationale for including literary and 
informational text on the NAEP Reading Assessment, and this chapter begins by describ- 
ing the text structures and features and aspects of author’s craft about which items will be 
developed. 

The matrices in exhibits 3 and 4 show the kinds of literary and informational texts that 
will be sampled at grades 4, 8, and 12, along with the text structures and literary devices 
or elements of author’s craft about which items may be developed. 

The matrices are designed to show the following aspects of literary and infonnational 
text: 

• Genres and types of text to be assessed. 

• Text structures and features about which items may be asked. 

• Aspects of author’s craft about which items may be asked. 

Types of text refers to the idealized nonns of a genre (Fludemik 2000), not the source of 
the stimulus material per se. 

Text structures and features define the organization and elements within the text. The 
organization and elements refer to the ways ideas are arranged and connected to one 
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another. Features refer to visual and structural elements that support and enhance the 
reader’s ability to understand the text. 

Author’s craft pertains to the specific techniques that an author chooses to relay an 
intended message. 

Entries listed within each cell of the matrices should be construed as neither definitive 
nor inclusive of all text structures and features or techniques of author’s craft. However, 
it is important to delineate the type of text to be used in reading comprehension tests 
(Kobayashi 2002; Wixson and Peters 1987). Understanding the range of text types for 
inclusion in the NAEP Reading Assessment illuminates the complex nature of reading 
comprehension passages and the accompanying questions. Items will assess students’ ap- 
plication of knowledge about text types, text features and structures, and author’s craft, 
not their recognition of specific tenninology in isolation. The designation of entries in the 
matrices by grade level reflects the levels at which these components of text are presented 
in state English language arts standards. They have further been confirmed by expe- 
rienced teachers and teacher educators. 

Literary Text 



The literary text matrix shown in exhibit 3 outlines the common forms of continuous 
prose and poetry that may be included. The matrix is divided into three sections (fiction, 
literary nonfiction, and poetry) and provides information on the aspects of text about 
which items will be developed. Successively more complex text forms are added at each 
level. 2 



‘A detailed explanation of the literary and informational text matrices will be provided in the Specifications 
for the NAEP Reading Assessment. 
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Exhibit 3. Literary text matrix: Fiction 





Genre/Type of Text 


Text Structures and 
Features 


Author’s Craft 






• 


Adventure stories 


• Themes 


Diction and word choice 






• 


Flistorical fiction 


• Morals 


• 


Dialogue 






• 


Contemporary realistic 


• Lessons 


• 


Exaggeration 








fiction 


Organization 


• 


Figurative language 




Tt 


• 


Folktales 


• Plot: sequence of 




• Symbolism 




QJ 


• 


Legends 


events 




• Simile and meta- 




a 

u 

T H 


• 


Fables 


• Conflict 




phor 




o 


• 


Tall tales 


• Solution 










• 


Myths 


• Resolution 










• 


Fantasy 


Elements 














• Setting 














• Characterization 










• 


Science fiction 


Organization 


• 


Mood 










• Parallel plots 


• 


Imagery 










• Circular plots 


• 


Flashback 










Elements 


• 


Foreshadowing 




oo 

QJ 






• Point of view 


• 


Personification 


** 

# o 


CQ 






• Contradictions 








L* 

O 






• Internal vs. external 






E 








conflict 










Plus increasingly complex 


Plus increasingly complex 


Plus increasingly complex 






application of grade 4 


application of grade 4 


application of grade 4 






• 


Satire 


Organization 


• 


Dramatic irony 






• 


Parody 


• Differentiation of plot 


• 


Character foils 






• 


Allegory 


structures for different 


• 


Comic relief 






• 


Monologue 


purposes and au- 


• 


Unconventional use of 










diences 




language 










Elements 








QJ 






• Interior monologue 








CQ 

U 






• Unreliable narrators 








o 






• Multiple points of 














view 










Plus increasingly complex 


Plus increasingly complex 


Plus increasingly complex 






application of grades 4 and 


application of grades 4 and 


application of grades 4 and 






8 




8 


8 





Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 
features or techniques of author’s craft. 



Fiction 

As suggested in the matrix, students in elementary and middle schools read many 
different kinds of stories for enrichment and enjoyment. These texts are representative of 
the developing conceptual understandings fonned by students during this period. At 
grades 8 and 12, more complex genres of fiction are common including satires, parodies, 
science fiction, and allegories. For purposes of the NAEP Reading Assessment, these 
genres may be either intact passages or passages excerpted from longer genres such as 
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novels. Material excerpted from longer pieces will be carefully analyzed to ensure that it 
has the structural integrity and cohesion necessary to sustain item development. 



The matrix also shows the aspects of text structures and features and author’s craft that 
may be assessed. These components, as well as the purposes for reading, become increa- 
singly complex and sophisticated as students move through the elementary, middle, and 
high school grades. For example, themes may be more abstract; plots may involve inter- 
nal or external conflicts; characterization may develop with antagonists, protagonists, and 
narrators with intertwined motives, beliefs, traits, and attitudes; the theme and setting 
may be more integral to one another; the plot may consist of a series of rising and falling 
actions within episodes; and the point of view or vantage point chosen by the author to 
reveal ideas, characters, or actions becomes more sophisticated, often including a shifting 
point of view or multiple points of view. 

Authors select from a range of stylistic devices to enhance their presentation. In the 
matrix, these are referred to as author’s craft. At grade 4, author’s craft includes figura- 
tive language such as symbolism, simile, metaphor, diction and word choice, dialogue, 
and exaggeration. More abstract elements, such as flashback and imagery, are part of au- 
thor’s craft at grade 8 in addition to more complex applications of the types of author’s 
craft listed for grade 4. Fictional passages for grade 12 are complex and may include the 
following literary devices — dramatic irony, character foils, comic relief, and unconven- 
tional use of language — in addition to the devices listed at grades 4 and 8. 
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Exhibit 3 (continued). Literary text matrix: Literary nonfiction 





Genre/Type of Text 


Text Structures and 
Features 


Author’s Craft 






• Personal essay 


Organization 


• 


Diction and word 






• Autobiographical and 


• Description 




choice 






biographical sketches 


• Cause and effect 


• 


Use of exposition. 








• Comparison 




action, or dialogue to 




QJ 




• Chronology 




introduce characters 




TS 

a 




Elements 


• 


Exaggeration 




u 

O 




• Point of view 


• 


Figurative language 








• Themes or central ideas 




• Symbolism 


st 






• Supporting ideas 




• Simile and 


w 

*43 






• Logical connections 




metaphor 


U 

« 






• Transitions 






a 

o 




Character sketch 




• 


Voice 


z 




• Memoir 




• 


Tone 


>. 

u 

0m* 


0© 


• Speech 




• 


Imagery 


u 

QJ 


QJ 






• 


Metaphoric language 


3 


u 

o 






• 


Irony 






Plus increasingly complex 


Increasingly complex 


Plus increasingly complex 






application of grade 4 


application of grade 4 


application of grade 4 






• Classical essay 




• 


Denotation 










• 


Connotation 




GJ 

a 

u 


Plus increasingly complex 


Increasingly complex 


Plus increasingly complex 




O 


application of grades 4 and 
8 


application of grade 4 


application of grades 4 and 
8 



Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 
features or techniques of author’s craft. 



Literar y Nonfiction 

The second type of literary text is literary nonfiction, which may include elements of 
narration and exposition and is often referred to as mixed text (Alexander and Jetton 
2000). Literary nonfiction is an example of mixed text because it uses literary techniques 
usually associated with fiction or poetry and also presents infonnation or factual material. 
Stylistically, it frequently blends literary elements and devices with factual infonnation 
with the dual purpose of infonning and offering reading satisfaction. Text types for lite- 
rary nonfiction at grade 4 include autobiographical and biographical sketches, and 
personal essays. At grade 8, additional forms of literary nonfiction used include character 
sketches, memoirs, and speeches. Classical essays are introduced as literary nonfiction at 
grade 12. Unlike texts that can be categorized as informational because of their sequen- 
tial, chronological, or causal structure, literary nonfiction uses a storylike structure. 
Classical essays may interweave personal examples and ideas with factual information to 
attain their purpose of explaining, presenting a perspective, or describing a situation or 
event. 
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Literary nonfiction selected for inclusion on NAEP will conform to the highest standards 
of literary quality. Literary nonfiction combines structures from both literary and infor- 
mational texts. At grade 4, text structures and features in literary nonfiction include 
description, cause and effect, comparison, chronology, point of view, themes and central 
ideas, and supporting ideas. At grades 8 and 12, increasingly complex structures listed 
above are noted in literary nonfiction. Text features such as logical connective devices 
and transitional devices are listed at grade 4. 

A range of literary devices and techniques termed author’s craft are present in literary 
nonfiction. Examples of author’s craft at grade 4 include diction and word choice, various 
ways to introduce characters, exaggeration, and figurative language. At grade 8, increa- 
singly complex techniques are listed for author’s craft: voice, tone, imagery, metaphoric 
language, and irony. Denotation and connotation are listed at grade 12 for author’s craft. 
Grades 8 and 12 will include more complex fonns of the text structures and features and 
author’s craft listed at grade 4. 
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Exhibit 3 (continued). Literary text matrix: Poetry 





Genre/Type of Text 


Text Structures and 
Features 


Author’s Craft 






• 


Narrative poem 


Organization 


• Diction and word choice 






• 


Lyrical poem 


• Verse 


(including the decision to 






• 


Humorous poem 


• Stanza 


omit words that may 






• 


Free verse 


Text features 


leave the reader with 










• Repetition 


much to infer) 










• Omission 


• Choice of different forms 










• Dialogue 


of poetry to accomplish 




Tf 

QJ 






• Line organization 


different purposes 




-d 

cg 






• Patterns 


• Exaggeration 




u 

o 






Elements 


• Use of imagery to pro- 










• Rhyme scheme 


vide detail 










• Rhythm 


• Figurative language 










• Mood 


• Simile 










• Themes and intent 


• Metaphor 












• Imagery 












• Alliteration 


>> 










• Onomatopoeia 


u 




• 


Ode 


Elements 


• Figurative language 


o 




• 


Song (including bal- 


• Abstract theme 


• Symbolism 




0© 




lad) 


• Rhythm patterns 


• Personification 




QJ 

TJ 

CG 


• 


Epic 


• Point of view 






5 






Plus increasingly complex 


Plus increasingly complex ap- 






Plus increasingly complex 


application of grade 4 


plication of grade 4 






application of grade 4 










• 


Sonnet 


Elements 


• Denotation 






• 


Elegy 


• Complex themes 


• Connotation 










• Multiple points of 


• Irony 




r4 






• view 


• Tone 




QJ 






• Interior monologue 


• Complex symbolism 




-d 

CG 






• Soliloquy 


• Extended metaphor and 




u 

o 






• Iambic pentameter 


analogy 






Plus increasingly complex 


Plus increasingly complex 


Plus increasingly complex ap- 






application of grades 4 and 
8 


application of grades 4 and 
8 


plication of grades 4 and 8 



Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 
features or techniques of author’s craft. 



Poetry 

The third type of literary text included in the NAEP Reading Assessment is poetry. Like 
fiction, poetry has distinctive forms, functions, and structures further guided by literary 
structures and textual features. The matrix lays out the kinds of poetry that students 
encounter at different grade levels. Thus, basic poetic fonns at grade 4 are narrative, 
lyrical, and humorous poems and free verse. Additionally at grade 8, odes, songs, and 
epics are included in the matrix for possible item development. More complex poetic 
forms are included at grade 12, such as sonnets and elegies. It is possible that two poems 
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may be used together in intertextual item sets to allow students to perfonn complex read- 
ing tasks, such as comparing thematic treatment in two poems or contrasting two poets’ 
choices of literary devices. 

Readers use the structure of poetry to aid in comprehension. Poetic structures range from 
simple to complex. Students at grade 4 can be expected to be familiar with simple organi- 
zational patterns such as verse and stanza along with the basic elements of rhyme 
scheme, rhythm, mood, and themes and intent. At grades 8 and 12, increasingly complex 
poetic organizational patterns and elements will be included. Students will also be 
expected to understand the use of “white space” as a structural feature of poetry. 

Understanding a poet’s choices also aids in understanding poetry. Language choice is of 
particular importance because the meaning in poetry is distilled in as few words as possi- 
ble. Poets choose from among a range of rhetorical structures and figurative language, 
using, for example, repetition, dialogue, line organization and shape, patterns, and many 
forms of figurative language. Increasingly complex application of figurative language, 
rhetorical devices, and complex poetry arrangements are included at grades 8 and 12. 

Informational Text 



As stated in chapter one, infonnational text on the NAEP Reading Assessment will be of 
three types: exposition, argumentation or persuasive text, and procedural text or docu- 
ments. Exhibit 4 presents the ways infonnational text will be assessed at grades 4, 8, and 
12. The matrix consists of three parts, each of which is accompanied by explanatory text. 
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Exhibit 4. Informational text matrix; Exposition 





Genre/Type of Text 


Text Structures and 
Features 


Author’s Craft 






• 


Informational trade 


• Organization 


• 


Transitional words 








book 


• Description 


• 


Signal words 






• 


Textbook 


• Sequence (e.g., 


• 


Voice 






• 


News article 


enumeration. 


• 


Figurative language and 






• 


Feature article 


chronology) 




rhetorical structures 






• 


Encyclopedia entry 


• Cause and effect 




• Parallel structure 






• 


Book review 


• Problem and solution 




• Quotations 










• Comparison and 




• Examples 










contrast 




• Repetition 




Tf 






Content features 




• Logical arguments 




QJ 






• Point of view 








a 

u 






• Topics or central ideas 








O 






• Supporting ideas and 














evidence 














Graphic features 














• Titles 














• Subheadings 














• Italics 






S3 








• Captions 














• Sidebars 






"S 








• Photos and illustrations 






o 

ft 








• Charts and tables 






x 

w 




• 


Flistorical document 




• 


Irony 






• 


Essay (e.g., 




• 


Sarcasm 




X 




informational, 










QJ 




persuasive, analytical) 










d 

u 

o 


• 


Research report 












Plus increasingly complex 


Increasingly complex 


Plus increasingly complex 






application of grade 4 


application of grade 4 


application of grade 4 






• 


Essay (e.g., political, 




• 


Denotation 








social, historical. 




• 


Connotation 








scientific, natural 




• 


Complex symbolism 








history) 




• 


Extended metaphor and 




fN 


• 


Literary analysis 






analogy 




QJ 

T3 








• 


Paradox 




a 

u 








• 


Contradictions/ 




0 








• 


incongruities 












• 


Ambiguity 






Plus increasingly complex 


Increasingly complex 


Increasingly complex 






application of grades 4 and 8 


application of grade 4 


application of grades 4 and 8 



Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 
features or techniques of author’s craft. 
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Exposition 



As they progress beyond the early grades, students read expository text with increasing 
frequency both in and out of school (Broer, Aarnoutse, Kieviet, and Van Leeuwe 2002). 
The primary goals of expository text for school-age readers are to communicate informa- 
tion and to advance learning. Fonns that may be assessed at grade 4 are informational 
trade books, textbook passages, news stories, feature stories, and encyclopedia entries. At 
grade 8, expository text genres include historical documents, various grade-appropriate 
essays, and research reports. More complex essay formats will be included for assessment 
at grade 12 such as political, social, historical, or scientific essays that primarily commu- 
nicate infonnation. 

Expository texts are characterized by internal sets of “grammars” similar in function to 
the story grammars discussed in chapter one. These grammars are designed to move the 
exposition forward and to help the reader comprehend the text. As shown in the matrix, 
the major organizational structures of exposition are description, sequence, cause and 
effect, problem and solution, and comparison and contrast (Meyer 1975). As mentioned 
in chapter one, exposition may also include lists as a structural component with lists of 
descriptions, causes, problems, solutions, and views presented within other structures. 
Commonly, exposition does not contain just one structural fonnat, but rather combines 
several structures embedded in the text. 

Specific elements within these organization structures signal meaning to the reader. 
Sequence, point of view, topics or central ideas, and supporting ideas and evidence are 
listed at grade 4; at grade 8 and grade 12, the structural organization and elements will be 
assessed at increasingly complex levels and with increasingly sophisticated texts. Some 
surface-level or graphic features support the text structures of exposition and guide the 
reader through the text. Other textual features can be categorized as reflecting author’s 
craft; these features guide the reader through the use of transitional words, signal words, 
voice, figurative language, and rhetorical structures. At grades 8 and 12, increasingly 
complex use of these features and of the author’s craft will be included for assessment. 
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Exhibit 4 (continued). Informational text matrix; Argumentation and persuasive text 





Genre/Type of Text 


Text Structures and 
Features 


Author’s Craft 






• Informational trade book 


Organization 


• Transitional words 






• Journal 


• Description 


• Signal words 






• Speech 


• Sequence (e.g.. 


• Voice 






• Simple persuasive essay 


enumeration. 


• Figurative language and 








chronology) 


rhetorical structure 








• Cause and effect 


• Parallel structure 








• Problem and solution 


• Quotations 








• Comparison and contrast 


• Examples 








Content features 


• Repetition 








• Author’s perspective or 


• Exaggeration 








position 


• Emotional appeal 








• Topics or central ideas 


• Tone 




Tf 




• Supporting ideas and 






a> 




evidence 






« 
Sh 
r h 




• Contrasting viewpoints 






w 




and perspectives 




QJ 

H 






• Presentation of the 




QJ 






argument (e.g., issue 




> 

l 






definition, issue choice. 




& 

a 






stance, relevance) 




u 






Graphic features 




QJ 

CLh 






• Titles 




'd 






• Subheadings 




G 

eg 






• Italics 




G 

o 






• Captions 




*43 






• Sidebars 




eg 

a 






• Photos and illustrations 




a 






• Charts and tables 




G 

g 




• Letter to the editor 




• Irony 


OX) 

Sh 




• Argumentative essay 




• Sarcasm 


< 


o© 


• More complex 




• Figurative language and 




QJ 


persuasive essay 




rhetorical structure 




Sh 


• Editorial 




• Parallel structure 




o 






• Quotations 






Plus increasingly complex 


Increasingly complex 


Plus increasingly complex 






application of grade 4 


application of grade 4 


application of grade 4 






• Essay (e.g., political. 










social) 










• Historical account 








fN 


• Position paper (e.g., 








a> 

TS 


persuasive brochure. 








cd 

Sh 

r h 


campaign literature, 








W 


advertisements) 










Plus increasingly complex 


Increasingly complex 


Increasingly complex 






application of grades 4 and 8 


application of grade 4 


application of grades 4 and 8 



Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 
features or techniques of author’s craft. 
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Argumentation and Persuasive Text 

Many forms of informational text pose an argument or attempt to persuade readers 
toward a particular viewpoint. These texts present information to support or prove a 
point, to express an opinion, and to try to convince readers that a specific viewpoint is 
correct or justifiable. Various logical fallacies and forms of bias may be found in argu- 
mentation and persuasive text. As the matrix shows, there is considerable similarity in 
structure, literary features, and elements among exposition, argumentation, and persua- 
sive text. The real distinction lies in the purpose for which an author writes these particu- 
lar kinds of informational text; as stated, exposition seeks to inform and educate, whereas 
argumentation and persuasive text seek to influence their readers’ thinking in other, often 
subtle but significant ways. 

At grade 4, argumentation and persuasive texts listed in the matrix are informational trade 
books that specifically argue a position or persuade the reader toward a stance, journals, 
speeches, and simple persuasive essays. However, in 201 1 NAEP will not assess argu- 
mentation and persuasive texts at grade 4 due to difficulty in locating high-quality texts 
appropriate for this grade level. At grade 8, there are more complex forms of argumenta- 
tion and persuasive texts: letters to the editor and editorials and argumentative and grade- 
appropriate persuasive essays. At grade 12, argumentation and persuasive texts become 
increasingly complex with a variety of types of essays such as political and social com- 
mentary essays, historical accounts, and position papers such as persuasive brochures, 
campaign literature, and advertisements. 

Particular organization techniques and elements are used to create a clear argument or to 
form a persuasive stand. The differences between exposition and argumentation and 
persuasive text lie not in the structural organization, but rather in the way the texts are 
elaborated through the use of contrasting viewpoints, shaping of arguments, appeals to 
emotions, and other manipulations of the elements of text and language. The organiza- 
tional structures at all levels are the same as in exposition: description, sequence, cause 
and effect, problem and solution, and comparison and contrast; they are represented in 
grades 8 and 12 with increasing complexity. 

Elements within these organizational structures include the author’s perspective, topics or 
central ideas, supporting ideas, contrasting viewpoints or perspectives, and the presenta- 
tion of the argument (e.g., issue definition, issue choice, stance, and relevance). These 
elements appear at all grade levels with complexity increasing at higher grade levels. In 
addition, at grade 12 students may be asked about the structure of a given argument; con- 
nections among evidence, inferences, and claims; and the structure of a deductive versus 
inductive argument. Twelfth grade students may also be asked questions about the range 
and quality of evidence, and logical fallacies, false assumptions/ premises, loaded terms, 
caricature, leading questions, and faulty reasoning in argumentation and persuasive texts. 
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Exhibit 4 (continued). Informational text matrix; Procedural texts and documents 





Genre/Type of Text 


Text Structures and Text Features 






Embedded in text 


Organization 






• Directions 


• Description 






• Map 


• Procedures 






• Timeline 


• Sequence (e.g., enumeration. 






• Graph 


chronology) 




Tf 


• Table 


Graphic features 




QJ 

d 


• Chart 


• Titles 


C/3 


a 

u 




• Labels 


s 

a» 


o 




• Headings 


S 

s 






• Subheadings 


o 

© 






• Sidebars 


Q 






• Photos and illustrations 


-o 

a 






• Charts and graphs 


a 






• Legends 


t/1 

+* 

X 




Embedded in text 




QJ 

H 


00 


• Recipe 




13 


QJ 

d 


• Schedule 




u 


a 

u 






F d 

QJ 


o 


Plus increasingly complex application of 




O 




grade 4 


Increasingly complex application of grade 4 


U 

CL, 




Stand-alone material 








• Application 






fN 


• Manual 






QJ 

d 


• Product support material 






a 

u 


• Contract 






o 










Plus increasingly complex application of 


Increasingly complex application of grades 






grades 4 and 8 


4 and 8 



Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 
features or techniques of author’s craft. 



Procedural Texts and Documents 

Research indicates that adults spend considerably more time reading documents (i.e., 
information in matrix or graphic form) than they do reading prose materials (Guthrie and 
Mosenthal 1987; Kirsch and Mosenthal 1990; Mosenthal 1996; Mosenthal 1998). Docu- 
ments and procedural texts are indeed common in our society; for example, we interpret 
bus schedules, assemble simple devices, order goods from a catalog, or follow directions 
to set the VCR clock. Such texts are used frequently in elementary and secondary 
schools, where students encounter textbooks replete with graphs, tables, and illustrations 
to accompany and expand traditional continuous text. 

Procedural text may be primarily prose arranged to show specific steps toward accom- 
plishing a goal or it may combine both textual and graphic elements to communicate with 
the user. Documents, in contrast, use text sparingly, in a telescopic way that minimizes 
the continuous prose that readers must process to gain the information they need. 
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As the matrix shows, document texts on the NAEP Reading Assessment may include, but 
are not limited to, tables and charts. Stand-alone procedural text or documents will not be 
included at grades 4 and 8; such text will be embedded in or ancillary to continuous text. 
They may appear as stand-alone stimuli at grade 12 but their use will account for only a 
small amount of the stimuli in the entire assessment. It is likely that many of the docu- 
ments may be used as part of intertextual item sets. For example, a student might encoun- 
ter a bar graph and a timeline with items that relate to both texts. 

Documents and procedural text features act as necessary clues to the organization of the 
text. As textual supports, these features guide the reader through the text. For the purpos- 
es of the NAEP Reading Assessment, graphic features include titles, labels, headings, 
subheadings, sidebars, photos and illustrations, charts and graphs, and legends at grades 
4, 8, and 12. More complex examples of these will be included at each successive grade. 

Characteristics of Texts Selected for Inclusion 



Passages selected as stimulus material for the NAEP Reading Assessment must meet 
rigorous criteria. They will all be authentic texts of the highest quality, evidencing cha- 
racteristics of good writing, coherence, and appropriateness for each grade level. Passag- 
es will be drawn from a variety of contexts familiar to students nationwide. Stimulus ma- 
terial must be engaging to students at each grade level. Furthermore, material must reflect 
our literary heritage by including recognized works from varied historical periods (Ra- 
vitch 2003). 

It is true that children’s experience differs from that of adults, and 
therefore the application of standards should be consonant with 
child life. Nevertheless, one must keep in mind the emotional 
maturity of the children for whom the book or books are intended. 

This does not mean that the works must be watered down so as to 
meet the reading ability levels of young children. On the contrary, 
some books of lasting value outstrip their vocabulary lists and con- 
nect with children on emotional-maturity levels so that they can be 
understood and enjoyed by the young themselves. . . . [T]he stan- 
dards basic to good writing in adult literature are also basic to good 
writing for children (Georgiou 1988). 

Most material included on the assessment will be presented in its entirety as students 
would encounter it in their own reading. However, some material may be excerpted, for 
example, from a novel or a long essay. Excerpted material will be carefully analyzed to 
ensure that it is coherent in structure. 

Passage Length 

Material on the assessment will be of differing lengths as shown in exhibit 5. 



NAEP 2011 Reading Framework 
28 




Exhibit 5. Passage lengths for grades 4, 8, and 12 



Grade 


Range of Passage Lengths 
(Number of Words) 


4 


200-800 


8 


400-1,000 


12 


500-1,500 



Passages of these lengths are used for several reasons. To gain the most valid infonnation 
about students’ reading, stimulus material should be as similar as possible to what 
students actually encounter in their in-school and out-of-school reading. Unlike many 
common reading tests that use short passages, NAEP will present longer material that 
challenges students to use their strategic reading skills in ways that reflect the kinds of 
reading they do in nontest situations (Paris, Wasik, and Turner 1991). Furthermore, short 
passages usually will not yield approximately 10 distinct items, the required minimum 
number for each NAEP item set. Longer passages, with clear structural patterns, can 
support the development of multiple, distinct, nontrivial items that cover the range of 
content included in the literary and infonnational text matrices. These items will also al- 
low broad coverage of the cognitive targets discussed later in this chapter. 

It is expected that in some cases, two poems will be used together to assess students’ abil- 
ity to compare them in terms of their themes and stylistic features. Prose passages used in 
intertextual item sets will also be fairly short. Likewise, it is possible that two documents 
might be included as intertextual stimuli at grade 12. 

Selection of Literary and Informational Passages 

Several methods of evaluating passages will be used to ensure that the best possible sti- 
mulus material is included. Authentic material must be of the highest quality, and it must 
come from authentic sources such as those students would encounter in their 
in-school and out-of school reading. Material must be coherent and allow items that as- 
sess domain-specific knowledge (Kobayashi 2002). Additionally, systematic efforts will 
be made to ensure that texts selected for inclusion will of interest to the widest number of 
students. Readers become more engaged in text and consequently comprehend a selection 
better when they find the material interesting (Bauman 1986; Wade, Buxton, and Kelly 
1993; Wade and Moje 2000; Wade et al. 1993). Texts will reflect literary heritage by 
representing varied historical periods. 

Passages selected for inclusion on the assessment will be well written, interesting to read, 
and considerate; that is, easily comprehensible because they are well organized, have 
appropriate vocabulary, and, where needed, have useful supplemental explanatory fea- 
tures such as definitions of technical terms or topographical features. Ideas marked by 
graphic features such as italics, bold print, and signal words and phrases tend to be 
processed more easily and recalled longer than unmarked information. In selecting pas- 
sages, attention will be paid to written clues within text that can help readers understand 
structure, guide the development of main ideas, and influence the recall of information. 
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For example, readers tend to organize and remember emphasized infonnation better when 
authors lead them with signal words indicating main ideas (for example, the most impor- 
tant point here), with phrases indicating sequencing (such as words like first, second, 
third), and with statements cross-referencing disparate parts of text (Armbruster 1984). 

Especially in the selection of informational text, the degree of content elaboration will be 
an important criterion for passage selection. Sufficient elaboration of new concepts is 
needed if students are to gain sufficient information to respond to questions. Tersely writ- 
ten infonnational text tends to be more difficult for students to comprehend compared 
with text written with more elaborated explanations. Whether text is tersely written or 
presents fully elaborated content is particularly important with topics that may be beyond 
the background knowledge of some students. 

An inviting writing style can also enhance interest and thereby increase comprehension. 
Material may be interesting not because of what is said but because of how it is said. For 
example, writers can increase interest by using active rather than passive verbs, by 
including examples that make the writing less abstract, and by using vivid and unusual 
words. An inviting writing style also influences voice. Voice, the qualities that help a 
reader view text as communication between an author and a reader, can have a positive 
effect on recall (Beck, McKeown, and Worthy 1995). 

Expert judgment will be the primary method for evaluating and selecting passages for 
inclusion on the assessment. Additional methods include the use of story and concept 
mapping and vocabulary mapping. At least two research-based readability fonnulas will 
also be used to gather additional infonnation about passage difficulty (Klare 1984; White 
and Clement 2001). Passages will be thoroughly reviewed for potential bias and sensitivi- 
ty issues. 

Story and concept mapping procedures have been used to identify appropriate passages 
for previous assessments (Wixson and Peters 1987). These procedures result in a graphic 
representation of a possible stimulus selection that clearly highlights the hierarchical 
structure and the interrelatedness of the passage components. Story mapping, for exam- 
ple, will show how the setting of a story is related, and contributes to, the development of 
plot and theme. Concept mapping shows the structure of informational text along with the 
concepts presented and the relational li nk s among concepts. Organizing information hie- 
rarchically within a passage allows the identification of various levels of infonnation 
within a text so that items can target the most important aspects of what students read. 

As NAEP begins to assess vocabulary in a systematic way, the story and concept map- 
ping procedures will be modified to ensure that appropriate words are selected for item 
development. 
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Selection of Poetry 



In selecting poetry for the NAEP Reading Assessment, it will be important to detennine 
that potential poems present a theme instead of stressing primarily the melodic or stylistic 
aspects of language use. Especially at grades 4 and 8, the theme should be implicitly 
presented in terms that are not so abstract that they are beyond students’ comprehension. 
Words and phrases should be used with economy to support and amplify the meaning 
inherent in the text; the style should be distinguished by author’s craft and project the 
poet’s feelings about his or her topic or theme. The ideas presented must be accessible to 
students and it must be clear that poetry, rather than prose, is the better mode for present- 
ing these ideas. A good question to ask in selecting poetry is: 

Does the poetry, through its expression of theme and ideas, carry children 
beyond their immediate experiential level to extensions where language 
and imagination meet? (Georgiou 1988) 

Selection of Noncontinuous Text and Documents 

In addition to continuous text prose and poetry, the assessment will include prose aug- 
mented by noncontinuous textual elements such as embedded tables or charts. It will also 
include stand-alone documents at grade 12. An analysis of layout will be essential to 
ensure that embedded noncontinuous text is used appropriately in a way that is well inte- 
grated into the prose text and not gratuitously distracting. Equally, stand-alone documents 
must be rich with appropriate information about which questions can be asked. The num- 
ber of categories of information presented graphically and the clarity of the layout of 
documents will be essential criteria for selecting documents for inclusion. The vocabulary 
and concept load of multimedia elements and of documents will also be considered. 

Exhibit 6 summarizes the considerations for selecting passages and documents. The first 
two columns present considerations for literary and infonnational continuous text. The 
third column presents considerations that must be made in selecting noncontinuous text 
that is embedded within continuous text or documents that will be used as stand-alone 
stimulus material at grade 12. Certain considerations are considered essential for each 
kind of stimulus material and represent the fundamental characteristics that make a text 
or document appropriate for inclusion. All potential stimulus material must also be 
grade-appropriate to ensure that students will be able to understand the concepts pre- 
sented and are familiar with the material’s stylistic features. Finally, balance must be 
considered so that the assessment as a whole reflects the full range of print and non- 
continuous text that students encounter in their in-school and out-of-school reading. 
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Exhibit 6. Considerations for selecting stimulus material 



Literary Text 


Informational Text 


Graphical Displays of 
Information 


Essential characteristics 

• Ability to engage readers 

• Well-written, rich text 

• Recognized literary merit 

• Theme/topic appropriateness by 
grade level 

Grade appropriateness 

• Complexity of characters 

• Number of characters 

• Vocabulary 

• Sophistication in use of literary 
devices 

• Complexity of dialogue 

• Point of view 

• Complexity of theme 

• Multiple themes (major/minor) 

• Use of time (flashbacks, 
progressive/digressive) 

• Illustrations 

Balance 

• Reflective of our literary 
heritage 

• Style 

• Variety of sentence and 
vocabulary complexity 

• Appropriateness of mode (prose 
vs. poetry) 

• Classical as well as 
contemporary 

• Representative of varied 
historical periods, cultures, 
socioeconomic backgrounds, 
etc. 

• Genre 


Essential characteristics 

• Ability to engage readers 

• Well-written, considerate text 

• Coherence 

• Theme/topic appropriateness 
by grade level 

Grade appropriateness 

• Topic 

• Vocabulary 

• Concepts (number, 
familiarity, abstractness) 

• Curricular appropriateness at 
grade level 

• Integrity of structure 

• Types of adjunct aids 

• Explicitness of perspective 

• Style 
Balance 

• Varied content areas 

• Style 

• Genre 

• Variety of sentence and 
vocabulary complexity 

• Appropriateness of mode 


Essential characteristics 

• Coherence 

• Clarity 

• Relevance (when embedded) 

Grade appropriateness 

• Structural complexity 

• Topic 

• Vocabulary 

• Concepts (number, 
familiarity, abstractness) 

• Number of categories of 
information presented 

• Amount of information 
within categories 

Balance 

• Embedded documents 
balanced with stand-alone 
documents (at grade 12) 

• Format 



Vocabulary on the NAEP Reading Assessment 



In 201 1, there will be an assessment of vocabulary in the context of passages that 
students read. Vocabulary knowledge is considered to be one of the five essential compo- 
nents of reading as defined by the No Child Left Behind (NCLB) legislation. In this 
context, vocabulary is construed not as isolated word meanings but as real knowledge of 
vocabulary that can advance comprehension. 

NAEP will not test definitions in isolation from surrounding text; that is, students will not 
be assessed on their prior knowledge of definitions. The definition of meaning vocabu- 
lary will guide the development: 
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Meaning vocabulary is the application of one’s understanding of word 
meanings to passage comprehension. 

Importance of Vocabulary for Reading Comprehension 

The associations between vocabulary and learning to read and then between vocabulary 
and reading comprehension are well documented in research (Hart and Risley 1995). 3 
Studies have repeatedly shown that students’ vocabulary is a fundamental factor in their 
ability to comprehend what they read. Not knowing the meaning of words as used in a 
given text may result in decreased comprehension of that text. Comprehending any read- 
ing passage requires knowing the meaning of the important content-bearing words of that 
passage, but often, the meaning of many key words in a passage depends on an interac- 
tion of word meaning and passage meaning (Bauman, Kame’enui, and Ash 2002; Lan- 
dauer, Foltz, and Laham 1998). Because of this interaction, measurement of word mean- 
ing by NAEP should be integrated with the measurement of passage comprehension. 

Several major factors are known to affect readers’ comprehension of what they read and 
can highlight the connection between word meaning and passage meaning; these include: 

• The context for reading (e.g., for study, for skimming, for leisure). 

• Fluency in identifying the words of the text. 

• Background or domain knowledge of the content of the text. 

• Knowledge of “the sense of the meaning” of the words the author uses to convey 
important content (Miller 1991). 

• Comprehension monitoring. 

Reasons for Assessing Vocabulary on NAEP Reading 

The growing body of research documenting the link between vocabulary and reading 
comprehension provides a strong rationale for the inclusion of a systematic measure of 
vocabulary. Past assessments have included a few vocabulary test items, all of which 
measured vocabulary in context; however, the number of items was scant and there were 
no specific vocabulary-related criteria for selecting the items or distractors. Furthermore, 
NAEP reports provided no information about performance on those items or how vocabu- 
lary performance might be related to reading comprehension. This change for the NAEP 
Reading Assessment, then, is significant. All vocabulary items will function both as a 
measure of passage comprehension and as a test of readers’ specific knowledge of the 
word’s meaning as intended by the passage author. 

Measurement of Meaning Vocabulary 

Vocabulary items will be developed about the meaning of words as they are used in the 
context of the passages that students read. Students will not be asked to draw on their 



3 For a complete list of references substantiating vocabulary assessment, see the bibliography. 
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prior knowledge by providing a written definition of each word on a list or in a set of 
words. There are two reasons for this approach. First, knowledge as explicit as a written 
definition of a word is not the specific ability required for passage comprehension. In 
reality, readers may not be able to provide a complete definition of a word they encounter 
but do have enough of the sense of the word’s meaning as used in text that their compre- 
hension is not impeded. 

A second argument against demanding specific definitions is that word meaning often 
depends on the context in which the word appears. Finding out whether readers know one 
specific definition of a word will not indicate whether they understand that word as it is 
used in a given text. Indeed, there is evidence that readers who know one definition of a 
word but not the meaning in a given text try to alter the sense of the text in keeping with 
their known definition: leading, of course, to misunderstanding the text (Deegan 1995). 

In addition, writers often use words in a manner that goes beyond concrete, familiar defi- 
nitions, but do so in ways that skilled readers can interpret effectively. Jacques Barzun 
describes this: 

Language is not an algebra; that is, the symbols do not stay put, nor can 
they be carried from place to place with an assurance that their value will 
not change. If language were like an algebra there could be no poetry or 
other fiction, no diplomacy or intimate correspondence, no persuasion or 
religious literature. If language were like an algebra, uncomfortable would 
mean not able to be comforted, and a myriad other nuances of human feel- 
ings would have to remain unrecorded and unshared (Barzun 1975). 

Considerations for Selecting Vocabulary 

In selecting passages, test developers must create a “map” of the story or expository 
selection to identify a passage’s key features. This procedure has included identifying 
candidates for vocabulary items, but the process will be enhanced to ensure that passages 
contain enough candidate words or terms for item development. 

The intent of the vocabulary assessment is to determine whether readers know and under- 
stand the meanings of the words that writers use to convey new information or meaning, 
not to measure readers’ ability to leam new terms or words. Hence, the assessment will 
focus on words that characterize the vocabulary of mature language users and character- 
ize written rather than oral language. The words selected for item development will 
convey concepts, ideas, actions, or feelings that the readers most likely know. In general, 
the words selected as targets for item development characterize the language of mature 
readers and are used in texts from a variety of content domains (Beck, McKeown, and 
Kucan 2002). 4 Considerations for selecting words for item development are summarized 
in exhibit 7. 



^Referred to as "tier 2” words, a term that distinguishes them from tier 1 words, which are common, every- 
day words basic to the speech and writing of most students, and from tier 3 words, rarely used words or 
technical terminology. 
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Exhibit 7. Considerations for selecting vocabulary items and distractors 



Vocabulary Words To Be 
Tested 



Vocabulary Words 
Excluded From Testing 



Considerations for 
Distractors 



Characterize the 
vocabulary of mature 
language users and 
characterize written rather 
than oral language. 

Label generally familiar 
and broadly understood 
concepts, even though the 
words themselves may 
not be familiar to younger 
learners. 

Necessary for 
understanding at least a 
local part of the context 
and linked to central 
ideas such that lack of 
understanding may 
disrupt comprehension. 

Are found in grade-level 
reading material. 



Narrowly defined and not 
widely useful, such as 
those related to specific 
content domains (e.g., 
photosynthesis, fiduciary) 
or words with limited 
application (e.g., deserter, 
hamlet). 

Label or name the main 
idea of the passage (e.g., 
the word “emancipation” 
would not be tested in an 
article dealing with the 
“Emancipation 
Proclamation”). 

Already likely to be part 
of students’ everyday 
speaking vocabulary at 
grade level. 

Meanings readily derived 
from language context 
(e.g., appositives, 
parenthetic definitions, 
idiomatic expressions). 



Present a different 
common meaning of the 
target vocabulary word, 
which must be ignored in 
favor of the meaning in 
context. 

May present correct 
information or content 
from the text that is not 
what is meant by the target 
word. 

May be an alternative 
interpretation of the 
context in which the target 
word occurs. 

May be the meaning of 
another word that looks or 
sounds similar to the 
target word. 

May present a common 
but inaccurate association 
with the target word. 



Words that are appropriate for inclusion denote concepts or things that readers already 
know. That is, the word denotes an object, idea, feeling, or action that has been expe- 
rienced or has been seen by the readers. However, the test item is not designed to deter- 
mine whether readers know the thing, but rather whether readers are able to link this 
knowledge (object, idea, feeling, action) to the word the author uses to convey this mean- 
ing. NAEP presumes that most readers will likely have the background knowledge of the 
object, idea, feeling, or action in a passage, but because the words are difficult and 
uncommon, readers may not readily link that knowledge to the specific word the author 
uses to convey that meaning. If readers do not connect a meaning with the author’s word, 
their comprehension will suffer. NAEP vocabulary items are designed to test readers’ 
ability to connect an appropriate meaning to the candidate words to gain comprehension. 
Thus, test items will not target technical terms or words identifying the central idea(s) of 
the passage because those words often represent new knowledge, concepts, or conceptua- 
lizations for readers. Passage comprehension items will measure readers’ learning from 
text; vocabulary items will measure readers’ knowledge of certain important words the 
author uses to impart this meaning. 
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Clearly, some students will know and understand some test words before taking the 
assessment. This is unavoidable. Furthermore, we anticipate that some readers will not 
have the background to link to the author’s words and thus will either choose an incorrect 
response for the item because of their background knowledge or identify the meaning of 
the word from context and mark the correct response. Recognizing this possibility, NAEP 
will ensure that the vocabulary test items represent a continuum of difficulty across read- 
ers at a given grade (as will reading passages and comprehension items). The intent is to 
identify words that the majority of grade-level students do not generally use in speaking 
or writing, but have seen or heard at least a few times. 

Cognitive Targets 



Items will be developed to assess students’ comprehension of literary and informational 
text. The term cognitive targets refers to the mental processes or kinds of thinking that 
underlie reading comprehension. Test questions will be aligned to cognitive dimensions 
applicable to literary and informational texts and also to cognitive dimensions specific to 
each text type. The remainder of the chapter presents those cognitive dimensions targeted 
by the items (hence the tenn cognitive targets) and discusses the item types included on 
the assessment. Inclusion of specific cognitive targets reflects the intent of the definition 
of reading that guides the assessment. The definition, explained in chapter one, follows. 

Reading is an active and complex process that involves: 

• Understanding written text. 

• Developing and interpreting meaning. 

• Using meaning as appropriate to type of text, purpose, and situation. 

Reading Processes Included in Cognitive Target Matrices 

The reading processes included in the three sections of the cognitive target matrix, exhibit 
8, illustrate the complex nature of reading. The research literature contains numerous stu- 
dies that show how students use different reading processes when reading various types 
of text (see chapter one). Hence, the sections of the matrix representing literary and in- 
formational text emphasize that different texts elicit different kinds of reading behaviors. 
The reading processes presented in the matrix are also grounded in the research 
literature on comprehension, most specifically the literature that uses protocol analysis 
(“think-alouds”) as its research methodology (Garner 1982; Guthrie, Britten, and Barker 
1991; Norris and Phillips 1987; Pressley and Afflerbach 1995; Olvshavsky 1976-77). 
Furthennore, they reflect the cognitive processes assessed on international reading 
assessments such as the Progress in International Reading and Literacy Study (PIRLS) 
(Campbell et al. 2001) and the Programme for International Student Assessment (PISA) 
(OECD 2000). The behaviors presented in exhibit 8 are illustrative, not comprehensive. 
The Specifications for the NAEP Reading Assessment will provide a detailed listing of the 
cognitive targets for item development. 
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Locate and Recall 



The first cognitive behaviors are locate and recall. As students locate or recall informa- 
tion from what they read, they may identify clearly stated main ideas or supporting 
details or they may find essential elements of a story, such as characters, time, or setting. 
Their process in answering assessment items often involves matching infonnation given 
in the item to either literal or synonymous infonnation in the text before they can then use 
the textual information to develop a response. As readers engage in these behaviors, they 
monitor their reading in order to understand when they are comprehending and when they 
are not. When they realize that the text is not making sense, they employ specific strate- 
gies to ensure that they begin to comprehend again. 

A salient activity [in reading] is to find the main ideas in the text and make 
certain that these ideas are remembered — or at least can be found again if 
needed. The big ideas, of course, are always relative to the goals of the 
reading with respect to the text. That is, very different ideas may be consi- 
dered main ideas if a reader is reading for one purpose versus another 
(Pressley and Afflerbach 1995, p. 44). 

Items assessing this component of reading usually focus on information contained in rela- 
tively small amounts of text: a sentence, a paragraph, or two or more adjacent paragraphs. 
These items provide information about the most basic comprehension skills, those that 
ultimately fonn the foundation for a more elaborate understanding of what is read. At the 
same time, these items address the kinds of reading that occur routinely in school and in 
out-of-school reading activities. 

Regardless of a reader’s goal — whether reading is done in preparation for 
a test, in anticipation of a writing assignment, with the expectation of 
sharing it in a conversation, to determine an author’s perspective, or as 
part of staying abreast in an area of interest — it is necessary to identify the 
important information in a text (Pressley and Afflerbach 1995, p. 3 1). 

Integrate and Interpret 

The next set of reading behaviors refers to what readers do as they integrate new infor- 
mation into their initial sense of what a passage says, often interpreting what they read in 
the process. When readers engage in behaviors involving integrating and interpreting, 
they make comparisons and contrasts of infonnation or character actions, examine rela- 
tions across aspects of text, or consider alternatives to what is presented in text. This 
aspect of the reading is critical to comprehension and can be considered the stage in 
which readers really move beyond the discrete infonnation, ideas, details, themes, and so 
forth presented in text and extend their initial impressions by processing information log- 
ically and completely. As readers integrate information and interpret what they read, they 
frequently fonn questions, use mental images, and make connections that draw on larger 
sections of text, often at an abstract level. They also draw on their knowledge of the 
structure and elements of literary and informational text. 
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In applying these behaviors, readers invariably think across large portions of text, across 
the text as a whole, or even across multiple texts; they relate textual infonnation to know- 
ledge from other sources such as their previous content learning or to internalized criteria 
and logic. Thus, readers might ask themselves whether something they are reading makes 
sense to them within the realm of their own experiences or when considered against what 
they have read in other sources. They examine the text in terms of their 
specific reading goals or the needs they have for the infonnation that the text can provide. 
In certain reading situations, readers may apply what they know to what they are reading, 
for example, detennining a real-world application of suggestions in a text on bicycle safe- 
ty. They also apply information gained from reading, for example in following 
instructions for repairing a bicycle or reading a map to determine where bike routes have 
been designated in a city. 

Readers are aware of many different aspects of text and the reading task 
they are performing from the outset of reading. Their perceptions of the 
text and how it relates to their task/ reading goals does much to shape the 
processing of text, with readers processing some parts of the text superfi- 
cially and others very carefully. . . . Good readers not only know what they 
are doing but also why they are doing it, ever aware of the characteristics 
of text they are confronting and their own reading goals (Pressley and 
Afflerbach 1995, p. 68). 

Items assessing these behaviors might ask students to fonn generalizations about a piece 
of informational text or make statements about how the setting of a story contributes to 
the creation of theme. Other items might require interpretation, for example, of a charac- 
ter’s motivations or of an author’s reasons for attempting to persuade readers about an 
issue. Other questions might ask for alternative actions that a character might have taken 
or an interpretation of an implied message or moral from a story. 

Critique and Evaluate 

The final set of reading behaviors, critiquing and evaluating text, requires readers to 
stand back from what they read and view the text objectively. The focus remains on the 
text itself but the reader’s purpose is to consider the text critically by assessing it from 
numerous perspectives and synthesizing what is read with other texts and other expe- 
riences. Items may ask students to evaluate the quality of the text as a whole, to deter- 
mine what is most significant in a passage, or to judge the effectiveness of specific tex- 
tual features to accomplish the purpose of the text (e.g., the effectiveness of details se- 
lected to support a persuasive argument). Items might ask for the likelihood that an event 
could actually have taken place, the plausibility of an argument, or the adequacy of an 
explanation for an event. Items can ask students to focus at the level of language choices 
(for 

example, nuances expressed in a metaphor) or at the broader level of the entire text (for 
example, evaluating the effectiveness of an author’s craft to accomplish his or her overall 
goals). 
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To answer these questions, students draw on what they know about text, language, and 
the ways authors manipulate language and ideas to achieve their goals. 

Sometimes readers recognize from the very start that they are likely to be 
evaluative with respect to a text, and likely to react to it affectively. ... 

Although some readers evidence great consistency in their evaluative 
stances as they read some texts, evaluations are often much more discri- 
minated. Regardless of whether a reader is globally positive, globally neg- 
ative, or a mixture of both, evaluations occur with respect to the style and 
context of text (Pressley and Afflerbach 1995, p. 76). 

Assessing Cognitive Targets 

Exhibit 8 presents the cognitive target matrix for the development of items to be used on 
the NAEP Reading Assessment. ^ The tenn cognitive targets is used to refer to the mental 
processes or kinds of thinking that underlie reading comprehension; the cognitive targets 
serve to guide the test development process in that item writers “target” these processes 
or kinds of thinking as they write items. The cognitive targets remain the same across all 
three grades on the assessment, but the passages and documents about which items are 
developed will be of increasing sophistication at each grade. 



5 The cognitive targets matrix is for illustrative purposes only and should not be considered an exhaustive 
list. The cognitive targets will be elaborated further in the Specifications for the NAEP Reading Assessment. 
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Exhibit 8. Cognitive targets 
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Items will be developed to assess all cognitive targets at each grade level, but the distri- 
bution of cognitive targets will vary across grades. In detennining the distribution across 
grade levels, careful thought was given to the kinds of texts that students encounter at 
each level. Reference was also made to the distribution across reading processes in the 
two international reading assessments, PISA and PIRLS (Campbell et al. 2001; OECD 
2000). Exhibit 9 displays the distribution of cognitive targets across grades 4, 8, and 12. 



Exhibit 9. Percentage distribution of cognitive targets by grade 



Grade 


Locate/Recall 


Integrate/Interpret 


Critique/Evaluate 


4 


30 


50 


20 


8 


20 


50 


30 


12 


20 


45 


35 



Item Types 



The NAEP Reading Assessment will include multiple-choice and constructed-response 
items. Both item types yield valuable infonnation about students’ reading and allow a 
rich, full description of how the nation’s students approach different kinds of text. The 
inclusion of both types of items affirms the complex nature of the reading process 
because it recognizes that different kinds of information can be gained from each item 
type. It also acknowledges the real-world skill of being able to write about what one has 
read. 

Multiple-choice items will include four options: the right response and three incorrect 
responses. It is assumed that a multiple-choice item will take students approximately 1 
minute to complete. Short constructed-response items can be answered by one or two 
phrases or by one or two sentences; they should take students approximately 2 to 3 mi- 
nutes to complete. Extended constructed-response items should elicit longer, more elabo- 
rated answers of a paragraph or two. They should take students approximately 5 minutes 
to complete. Scoring rubrics for short and extended constructed-response items will focus 
on the content included in answers, not on spelling or grammatical considerations. How- 
ever, students must answer constructed-response questions by using infonnation from the 
text to receive credit. Details regarding the scoring and short and extended constructed- 
response items appear in the Specifications for the NAEP Reading Assessment 6 

The distribution of multiple-choice and constructed-response items will vary across the 
grades assessed by the NAEP Reading Assessment. The percentages in exhibit 10 refer to 
the amount of assessment time that students will spend responding to these particular 
kinds of items. Hence, grade 4 students will spend approximately 50 percent of the 



6 The specifications will guide the development of the assessment. It will provide detailed information about 
the kinds of reading selections to be included, item types, and scoring criteria for constructed-response 
items. The specifications will also discuss test administration procedures, any considerations to be given to 
special populations, and special studies to be conducted in conjunction with the assessment (see appendix 
C). 
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assessment time responding to multiple-choice items and 50 percent of the assessment 
time preparing written responses. Students at grades 8 and 12 will spend more time 
preparing written responses. 

Approximately two items per passage will assess vocabulary knowledge. These items 
may be either multiple choice or short constructed response in format. Exhibit 10 shows 
the distribution of time to be spent on each kind of item. 



Exhibit 10. Percentage distribution ol 


1 time to be spent on specific item types 


Grade 


Multiple Choice 


Short Constructed 
Response 


Extended Con- 
structed Response 


4 


50 


40 


10 


8 


40 


45 


15 


12 


40 


45 


15 



Less time is allocated to constructed-response items at grade 4 to reflect developmental 
differences. Students at grade 4 may not be as familiar with written responses to reading 
questions as are older students (Kobayashi 2002). 



NAEP 2011 Reading Framework 



42 





Chapter Three 



Reporting Results 

Results of the NAEP Reading Assessment administrations are reported in terms of 
average scores for groups of students on the NAEP 0-500 scale and as percentages of 
students who attain each of the three achievement levels (Basic, Proficient, and 
Advanced) discussed below. This is an assessment of overall achievement, not a tool for 
diagnosing the needs of individuals or groups of students. Reported scores are always at 
the aggregate level; by law, scores are not produced for individual schools or students. 
Results are reported for the nation as a whole, for regions of the nation, for states, and for 
large districts that volunteer to participate in the NAEP trial urban district assessment 
(TUDA). 

No Child Left Behind Provisions for NAEP Reporting 



Under the provisions of the No Child Left Behind (NCLB) legislation, states receiving 
Title I grants must include assurance in their state plans that they will participate in the 
reading and mathematics state NAEP at grades 4 and 8. Local districts that receive Title I 
funds must agree to participate in biennial NAEP administrations at grades 4 and 8 if they 
are selected to do so. Their results will be included in state and national reporting. Partic- 
ipation in NAEP will not substitute for the mandated state-level assessments in reading 
and mathematics at grades 3 to 8. 

In 2002, NAEP initiated TUDA in five large urban school districts that are members of 
the Council of Great City Schools (the Atlanta City, City of Chicago, Houston Indepen- 
dent, Los Angeles Unified, and New York City Public Schools districts). Ten large 
districts participated in 2003 and 2005. The number of districts participating in TUDA 
has grown over time. Districts that participate in TUDA receive their own data, which 
they can use for assessing the achievement of their own students and for comparative 
purposes. 

Achievement Levels 



Since 1990, the National Assessment Governing Board has used student achievement le- 
vels for reporting results on NAEP assessments. The achievement levels represent an in- 
formed judgment of “how good is good enough” in the various subjects assessed. 

Generic policy definitions for achievement at the Basic, Proficient, and Advanced levels 
describe in very general terms what students at each grade level should know and be able 
to do on the assessment. Reading achievement levels specific to the NAEP Reading 
Framework were developed to elaborate on the generic definitions. New reading-specific 
achievement level descriptors 



NAEP 2011 Reading Framework 
43 





replaced those aligned to the previous framework (NAGB 2003). Preliminary achieve- 
ment level descriptors were developed for the assessment as a whole and for the vocabu- 
lary component of the assessment. These preliminary achievement levels were used to 
guide item development and initial stages of standard-setting. The preliminary achieve- 
ment level descriptions were refined as a result of the achievement level setting process. 

Exhibit 1 1 presents the generic achievement level descriptors. See appendix B for the 
final achievement level descriptions. 



Exhibit 11. Generic NAEP achievement levels 



Achievement Level 


Policy Definition 


Advanced 


This level signifies superior performance. 


Proficient 


This level represents solid academic performance for each 
grade assessed. Students reaching this level have 
demonstrated competency over challenging subject matter, 
including subject-matter knowledge, application of such 
knowledge to real-world situations, and analytical skills 
appropriate to the subject matter. 


Basic 


This level denotes partial mastery of prerequisite knowledge 
and skills that are fundamental for proficient work at each 
grade. 



Reporting NAEP Results 



NAEP Reading Assessment results are reported in terms of average scores for groups of 
students on the NAEP 0-500 scale and as percentages of students who attain each of the 
three achievement levels (Basic, Proficient, and Advanced). Information is also provided 
about students who score below Basic. These students are not necessarily nonreaders; 
many can complete some tasks on the assessment but are not able to attain the minimum 
score required for Basic. 

Data are reported on subgroups of students by gender, race/ethnicity, eligibility for free 
or reduced-price lunch, region of the country, type of community, public or nonpublic 
school, and other variables of interest. Data are never provided for individual students or 
schools. Subscores should be provided for literary and infonnational texts. Results will 
also be provided about students’ responses to the vocabulary items. 

The 2011 results will continue to use a 0-500 cross-grade scale. Use of such a scale affirms 
that reading is a development process and that students’ reading skills mature throughout 
their school years as they read increasingly diverse and sophisticated texts. 

The primary vehicle for reporting NAEP reading results is the NAEP Reading Report 
Card issued after each assessment administration. The report provides detailed informa- 
tion on the assessments, the students who participated, and the assessment results. Results 
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are disaggregated by specific groups and are also presented for states that participate in 
the NAEP state assessment. Among the focal groups are males and females, students 
from various racial/ethnic backgrounds, and students who took the assessment with and 
without accommodations. 

NAEP data and infonnation about the assessments are also available electronically 
through the National Governing Assessment Board (www.nagb.org) and the National 
Center for Education Statistics/NAEP (nces.ed.gov) websites. Furthermore, the NAEP 
Data Explorer can be used by interested education administrators, researchers, and other 
stakeholders to develop focused reports. The NAEP e-Library (nces.ed.gov) provides 
access to other infonnation such as access to NAEP reports, sample assessment passages, 
items, scoring rubrics with student-constructed responses, and data sources for more 
in-depth analysis of student achievement results or of the assessments themselves. 

Reporting State NAEP Results 



As discussed above, states receiving Title I funding must participate in the NAEP Read- 
ing Assessment at grades 4 and 8. Results are reported in the aggregate for participating 
students and are also disaggregated for specific reference groups of students. Individual 
state reports are generated in addition to reports that contrast results from participating 
states and from the nation as a whole. The NAEP Report Generator allows state and local 
administrators and others to customize reports and to investigate specific aspects of 
student reading achievement. 

Reporting Trend Data 



According to NAEP law and Governing Board policy, long-tenn trend assessments are 
conducted as part of NAEP in order to continue the national trend reports. In reading, 
long-tenn assessments have been administered since 1971. The long-tenn trend reports 
provide the only continuous measures of student achievement over such extended periods 
of time. Passages and accompanying test items administered as part of the long-tenn 
trend assessments have remained unchanged from their initial administration in 1971. 

The 2009-2011 NAEP Reading Framework represents several important changes from 
the framework that has guided the assessment from 1992 to 2007 (see exhibit 2). The 
2011 NAEP Reading Assessment will use the same framework used in 2009. The Read- 
ing Framework for the 2009 National Assessment of Educational Progress replaced the 
framework first used for the 1992 reading assessment and then for subsequent reading 
assessments through 2007. Compared to the previous framework, the 2009 reading 
framework included more emphasis on literary and informational texts, a redefinition of 
reading cognitive processes, a new systematic assessment of vocabulary knowledge, and 
the addition of poetry to grade 4. 

The 2009 NAEP Reading Report Card included trend data on student reading perfor- 
mance from 1992 to 2009. Results from special analyses detennined the 2009 reading 
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assessment results could be compared with those from earlier assessment years. These 
special analyses started in 2007 and included in-depth comparisons of the frameworks 
and the test questions, as well as an examination of how the same students perfonned on 
the 2009 assessment and the earlier assessment. A summary of these special analyses and 
an overview of the differences between the previous framework and the 2009 framework 
are available on the web at http://nces.ed.gov/nationsreportcard/reading/ trend_study.asp. 
The 2011 NAEP Reading Report Card will report trends in student reading performance 
from 1992 to 2011. 

NAEP reports are useful in providing trend results over time to inform decisions and 
allocations of resources and framing of policy about reading. The questions that NAEP 
addresses include these: 



• Are students improving in reading achievement over time? 

• Are percentages of students at the upper achievement levels increasing, decreas- 
ing, or remaining the same? 

• Are the gaps in achievement among various groups narrowing? 

Assessments aligned to the 1992 framework and its subsequent versions have yielded 
trend data from seven national and six state administrations as shown in exhibit 14. 



Exhibit 12. Years of administration of NA 



IP reading assessments 



Year 


Grades for 
National 
Administration 


Grades for State 
Administration 


1992 


4, 8, 12 


4 


1994 


4, 8, 12 


4 


1998 


4, 8, 12 


4,8 


2000 


4 




2003 


4, 8, 12 


4,8 


2005 


4, 8, 12 


4,8 


2007 


4,8 


4,8 


2009 


4, 8, 12 


4, 8, 12 



Background Variables 

Students participating in the NAEP assessments respond to background questionnaires 
that gather infonnation on variables important to understanding reading achievement 
nationwide. Teachers and school administrators also complete background question- 
naires. To the extent possible, information is also gathered from non-NAEP sources such 
as state, district, or school records to minimize the burden on those asked to complete the 
questionnaires. Questions are nonintrusive; free from bias; secular, neutral, and nonideo- 
logical; and do not elicit personal feelings, values, or attitudes. 
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As stated in Governing Board policy, the collection of background data on students, 
teachers, and schools is necessary to fulfill the statutory requirement that NAEP include 
information whenever feasible that is disaggregated by race or ethnicity, socioeconomic 
status, gender, disability, and limited English proficiency. Background information serves 
the additional purpose of enriching the reporting of NAEP results by examining factors 
related to academic achievement in the specific subjects assessed. 

To satisfy the goal of enriching reports on student achievement in reading, background 
variables are selected to be of topical interest, timely, and directly related to academic 
achievement. The selection of variables about which questions will be developed may 
reflect current trends in the field, such as the use of technology in reading instruction or 
the extent to which students use the Internet as a reference tool. Recommendations on 
background variables for the NAEP Reading Assessment were presented as a separate 
document. 
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Appendix A 

Glossary of Terms 

This glossary provides brief definitions of terms used throughout the NAEP Reading 
Framework. The terms are defined according to their use in the framework. The list 
includes tenns that relate to types of reading materials, text structures and features, tech- 
niques of author’s craft, and other key terms. 

Allegory: Story in which the characters, settings, and events stand for abstract moral 
concepts. 

Alliteration: Repetition of initial consonant sounds. 

Allusion: Reference to a mythological, literary, or historical person, place, or thing. 

Analogy: Comparison of two things to show their likenesses in certain respects. 

Argumentation: Writing that seeks to influence through appeals that direct readers to 
specific goals or try to win them to specific beliefs. 

Audience: Writer’s targeted reader or readers. 

Author’s craft: Specific techniques that an author chooses to relay an intended message. 

Autobiography: Written account of the author’s own life. 

Ballad: Song or songlike poem that tells a story. 

Biography: Account of a person’s life written by another person. 

Causation: Text structure that presents causal or cause and effect relationships between 
the ideas presented in the text. 

Cognitive target: Mental process or kind of thinking that underlies reading comprehen- 
sion; cumulatively, the cognitive targets will guide the development of items for the as- 
sessment. 

Coherence: Continuity of meaning that enables others to make sense of a text. 

Comic relief: Event or character that serves as an antidote to the seriousness of dramatic 
events. 
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Comparison: Text structure in which ideas are related to one another on the basis of si- 
milarities and differences. The text presents ideas organized to compare, to contrast, or to 
provide an alternative perspective. 

Conflict: Struggle or clash between opposing characters, forces, or emotions. 

Connotation: Implicit rather than explicit meaning of a word. It consists of the sugges- 
tions, associations, and emotional overtones attached to a word. 

Description: Text structure that presents a topic, along with the attributes, specifics, or 
setting information that describe that topic. 

Denotation: Exact, literal definition of a word independent of any emotional association 
or secondary meaning. 

Detail: Fact revealed by the author or speaker that supports the attitude or tone in a piece 
of poetry or prose. In infonnational text, details provide information to support the au- 
thor’s main point. 

Diction: Word choice intended to convey a certain effect. 

Elegy: Poem that mourns the death of a person or laments something lost. 

Epic: Long narrative poem that relates the great deeds of a hero who embodies the values 
of a particular society. 

Exaggeration or hyperbole: Deliberate, extravagant, and often outrageous overstate- 
ment. It may be used for either serious or comic effect. 

Exposition: One of the classifications of discourse whose function is to infonn or to in- 
struct or to present ideas and general truths objectively. Exposition presents infonnation, 
provides explanations and definitions, and compares and contrasts. 

Fable: Brief story that teaches a moral or practical lesson about life. 

Fantasy: Story employing imaginary characters living in fictional settings where the 
rules of the real world are altered for effect. 

Fiction: Imaginative literary works representing invented rather than actual persons, 
places, and events. 

Figure of speech: Word or phrase that describes one thing in tenns of something else, 
often involving an imaginative comparison between seemingly unlike things. 

Flashback: Scene that interrupts the action of a work to show a previous event. 
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Fluency: Ability to read text quickly and accurately and comprehend what is read. 

Foil: Character who sets off another character by strong contrast. 

Folktale: Short story from the oral tradition that reflects the mores and beliefs of a 
particular culture. 

Foreshadowing: Use of hints or clues in a narrative to suggest future action. 

Free verse: Poetry that has no regular meter or rhyme scheme. 

Genre: Category used to classify literary and other works by fonn, technique, or content. 

Grammar: Coherent text structure on which readers rely as they seek to understand what 
they read; often referred to as “story grammar.” 

Graphic: Pictorial representation of data or ideas using columns, matrices, or other for- 
mats. Graphics can be simple or complex, present information in a straightforward way 
as in a list or pie graph, or embed or “nest” information within the document’s structure. 
Graphics may be included in texts or be stand-alone documents (grade 12 only). 

Historical fiction: Story that recreates a period or event in history and often uses histori- 
cal figures as characters. 

Iambic pentameter: Line of poetry made up of five metrical feet or units of measure, 
consisting of an unstressed syllable followed by a stressed syllable. 

Imagery: Multiple words or a continuous phrase that a writer uses to represent persons, 
objects, actions, feelings, and ideas descriptively by appealing to the senses. 

Inference: Act or process of deriving logical conclusions from premises known or as- 
sumed to be true; the conclusions drawn from this process. 

Irony: Tension that arises from the discrepancy, either between what one says and what 
one means (verbal irony), between what a character believes and what a reader knows 
(dramatic irony) or between what occurs and what one expects to occur (situational iro- 
ny). 

Legend: Inscription or title on an object (e.g., a key to symbols used on a map). 

Literary device: Literary technique used to achieve a particular effect. 

Literary heritage: Works by authors whose writing influenced and continues to influ- 
ence the public language, thinking, history, literary culture, and politics of this nation. 
These works comprise the literary and intellectual capital drawn on by later writers. 
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Literary nonfiction: Text that conveys factual infonnation. The text may or may not 
employ a narrative structure and characteristics such as dialogue. 

Lyrical poetry: Poems that focus on expressing emotions or thoughts. 

Meaning vocabulary: Application of one’s understanding of word meanings to passage 
comprehension. 

Memoir: Type of autobiography that usually focuses on a single time period or historical 
event. 

Metaphor: Comparison of two unlike things without the use of “like” or “as.” 

Mixed text: Text that employs literary techniques usually associated with narrative or 
poetry while also presenting infonnation or factual material, with the dual purpose of 
informing and offering reading satisfaction; requires readers to discern bias from fact. 

Monologue: Long, formal speech made by a character. 

Mood: Atmosphere or predominant emotion in a literary work. 

Motivation: Circumstance or set of circumstances that prompts a character to act a 
certain way or that determines the outcome of a situation or work. 

Myth: Traditional story accepted as history, which serves to explain the world view of a 
people. 

Narration: Telling of a story in writing. 

Narrative poetry: Poems that tell a story in verse, often focusing on a single incident. 

Ode: Long lyric poem on a serious subject often for ceremonial or public occasions. 

Onomatopoeia: Use of words that mimic the sounds they describe; imitative hannony. 

Parody: Imitation of a work of literature, art, or music for amusement or instruction. 

Parallel structure: Repetition of words, phrases, or sentences that have the same gram- 
matical structure or that restate a similar idea. 

Personification: Metaphor that gives inanimate objects or abstract ideas human characte- 
ristics. 

Perspective: Position, stance, or viewpoint from which something is considered or eva- 
luated. 
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Persuasion: Fonn of discourse whose function is to convince an audience or to prove or 
refute a point of view or an issue. 

Plot: Sequence of events or actions in a short story, novel, or narrative poem. 

Point of view: Perspective or vantage point from which a literary work is told or the way 
in which the author reveals characters, actions, and ideas. 

Problem/solution: Text structure in which the main ideas are organized into two parts: a 
problem and a subsequent solution that responds to the problem, or a question and an an- 
swer that responds to the question. 

Procedural text: Text that conveys infonnation in the fonn of directions for accomplish- 
ing a task. A distinguishing characteristic of such text is that it is composed of discrete 
steps to be perfonned in a strict sequence with an implicit end product or goal. 

Protagonist: Central character of a short story, novel, or narrative poem. The antagonist 
is the character who stands directly opposed to the protagonist. 

Purpose: Specific reason or reasons for the writing. It conveys what the readers have to 
gain by reading the selection. Purpose is the objective or the goal that the writer wishes to 
establish. 

Repetition: Deliberate use of any element of language more than once: sound, word, 
phrase, sentence, grammatical pattern, or rhythmical pattern. 

Rhetoric: Art of using words to persuade in writing or speaking. 

Rhetorical device: Technique used by writers to persuade an audience. 

Rhyme: Repetition of sounds in two or more words or phrases that appear close to each 
other in a poem. End rhyme occurs at the end of lines; internal rhyme, within a line. Slant 
rhyme is approximate rhyme. A rhyme scheme is the pattern of end rhymes. 

Rhythm: Regular recurrence and speed of sound and stresses in a poem or work of prose. 

Sarcasm: Use of verbal irony in which a person appears to be praising something but is 
actually insulting it. 

Satire: Prose in which witty language is used to convey insults or scorn. 

Sequence: Text structure in which ideas are grouped on the basis of order or time. 

Setting: Time and place in which events in a short story, novel, or narrative poem take 
place. 
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Simile: Comparison of two different things or ideas through the use of the words “like” 
or “as.” 

Sonnet: Fourteen-line lyric poem, usually written in iambic pentameter. 

Stanza: Division of a poem, composed of two or more lines. 

Style: Writer’s characteristic manner of employing language. 

Symbol: Object, person, place, or action that has both a meaning in itself and that stands 
for something larger than itself, such as a quality, attitude, belief, or value. 

Syntax: Arrangement of words and order of grammatical elements in a sentence. 

Tall tale: Improbable, incredible, or fanciful story. 

Theme: Central meaning of a literary work. A literary work can have more than one 
theme. Most themes are not directly stated but rather are implied. A literary theme is not 
the same as a topic. 

Tone: Writer’s or speaker’s attitude toward a subject, character, or audience conveyed 
through the author’s choice of words and detail. Tone can be serious, humorous, sarcas- 
tic, objective, etc. 

Trait: Distinguishing feature, as of a person’s character. 

Understatement: Kind of irony that deliberately represents something as being much 
less than it really is; the opposite of hyperbole or overstatement. 

Voice: Distinctive style or manner of expression of an author or of a character. 
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Appendix B 



NAEP Reading Achievement Level Definitions 

The National Assessment of Educational Progress (NAEP) reading achievement level 
descriptions present expectations of student performance in relation to a range of text 
types and text difficulty and in response to a variety of assessment questions intended to 
elicit different cognitive processes and reading behaviors. The specific processes and 
reading behaviors mentioned in the achievement level descriptions are illustrative of 
those judged as central to students’ successful comprehension of texts. These processes 
and reading behaviors involve different and increasing cognitive demands from one grade 
and perfonnance level to the next as they are applied within more challenging contexts 
and with more complex information. While similar reading behaviors are included at the 
different perfonnance levels and grades, it should be understood that these skills are be- 
ing described in relation to texts and assessment questions of varying difficulty. Bold text 
is a short, general summary to describe performance at each achievement level. 



NAEP Reading Achievement Levels — Grade 4 



Basic 

(208) 



Proficient 

(238) 



Fourth-grade students performing at the Basic level should be able to 
locate relevant information, make simple inferences, and use their un- 
derstanding of the text to identify details that support a given interpre- 
tation or conclusion. Students should be able to interpret the meaning 
of a word as it is used in the text. 

When reading literary texts such as fiction, poetry, and literary nonfiction, 
fourth-grade students performing at the Basic level should be able to make 
simple inferences about characters, events, plot, and setting. They should be 
able to identify a problem in a story and relevant information that supports 
an interpretation of a text. 

When reading informational texts such as articles and excerpts from books, 
fourth-grade students performing at the Basic level should be able to identi- 
fy the main purpose and an explicitly stated main idea, as well as gather in- 
formation from various parts of a text to provide supporting information. 

Fourth-grade students performing at the Proficient level should be able 
to integrate and interpret texts and apply their understanding of the 
text to draw conclusions and make evaluations. 

When reading literary texts such as fiction, poetry, and literary nonfiction, 
fourth-grade students performing at the Proficient level should be able to 
identify implicit main ideas and recognize relevant information that supports 
them. Students should be able to judge elements of author’s craft and 

provide some support for their judgment. They should be able to analyze 
character roles, actions, feelings, and motives. 
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Advanced 

(268) 



When reading informational texts such as articles and excerpts from books, 
fourth-grade students performing at the Proficient level should be able to 
locate relevant information, integrate information across texts, and evaluate 
the way an author presents information. Student performance at this level 
should demonstrate an understanding of the purpose for text features and an 
ability to integrate information from headings, text boxes, graphics and their 
captions. They should be able to explain a simple cause-and-effect relation- 
ship and draw conclusions. 

Fourth-grade students performing at the Advanced level should be able 
to make complex inferences and construct and support their inferential 
understanding of the text. Students should be able to apply their under- 
standing of a text to make and support a judgment. 

When reading literary texts such as fiction, poetry, and literary nonfiction, 
fourth-grade students performing at the Advanced level should be able to 
identify the theme in stories and poems and make complex inferences about 
characters’ traits, feelings, motivations, and actions. They should be able to 
recognize characters’ perspectives and evaluate character motivation. Stu- 
dents should be able to interpret characteristics of poems and evaluate as- 
pects of text organization. 

When reading informational texts such as articles and excerpts from books, 
fourth-grade students performing at the Advanced level should be able to 
make complex inferences about main ideas and supporting ideas. They 
should be able to express a judgment about the text and about text features 
and support the judgment with evidence. They should be able to identify the 
most likely cause given an effect, explain an author’s point of view, and 
compare ideas across two texts. 



NAEP Reading Achievement Levels — Grade 8 



isasic Eighth-grade students performing at the Basic level should be able to 

(243) locate information; identify statements of main idea, theme, or au- 

thor’s purpose; and make simple inferences from texts. They should be 
able to interpret the meaning of a word as it is used in the text. Students 
performing at this level should also be able to state judgments and give 
some support about content and presentation of content. 

When reading literary texts such as fiction, poetry, and literary nonfiction, 
eighth-grade students performing at the Basic level should recognize major 
themes and be able to identify, describe, and make simple inferences about 
setting and about character motivations, traits, and experiences. They should 
be able to state and provide some support for judgments about the way an 
author presents content and about character motivation. 

When reading informational texts such as exposition and argumentation, 
eighth-grade students performing at the Basic level should be able to recog- 
nize inferences based on main ideas and supporting details. They should be 
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Proficient 

(281) 



Advanced 

(323) 



able to locate and provide relevant facts to construct general statements 
about information from the text. Students should be able to provide some 
support for judgments about the way information is presented. 

Eighth-grade students performing at the Proficient level should be able 
to provide relevant information and summarize main ideas and themes. 
They should be able to make and support inferences about a text, con- 
nect parts of a text, and analyze text features. Students performing at 
this level should also be able to fully substantiate judgments about con- 
tent and presentation of content. 

When reading literary texts such as fiction, poetry, and literary nonfiction, 
eighth-grade students performing at the Proficient level should be able to 
make and support a connection between characters from two parts of a text. 
They should be able to recognize character actions and infer and support 
character feelings. Students performing at this level should be able to pro- 
vide and support judgments about character motivation across texts. They 
should be able to identify how figurative language is used. 

When reading informational texts such as exposition and argumentation, 
eighth-grade students performing at the Proficient level should be able to 
locate and provide facts and relevant information that support a main idea or 
purpose, interpret causal relations, provide and support a judgment about the 
author’s argument or stance, and recognize rhetorical devices. 

Eighth-grade students performing at the Advanced level should be able 
to make connections within and across texts and to explain causal rela- 
tions. They should be able to evaluate and justify the strength of sup- 
porting evidence and the quality of an author’s presentation. Students 
performing at the advanced level also should be able to manage the 
processing demands of analysis and evaluation by stating, explaining, 
and justifying. 

When reading literary texts such as fiction, literary nonfiction, and poetry, 
eighth-grade students performing at the Advanced level should be able to 
explain the effects of narrative events. Within or across text, they should be 
able to make thematic connections and make inferences about character 
feelings, motivations, and experiences. 

When reading informational texts such as exposition and argumentation, 
eighth-grade students performing at the Advanced level should be able to 
infer and explain a variety of connections that are intratextual (such as the 
relation between specific information and the main idea) or intertextual 
(such as the relation of ideas across expository and argument text). Within 
and across texts, students should be able to state and justify judgments about 
text features, choice of content, and the author’s use of evidence and rhetor- 
ical devices. 
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NAEP Reading Achievement Levels — Grade 12 



Basic 

(265) 



Proficient 

(302) 



Twelfth-grade students performing at the Basic level should be able to 
identify elements of meaning and form and relate them to the overall 
meaning of the text. They should be able to make inferences, develop 
interpretations, make connections between texts, and draw conclusions; 
and they should be able to provide some support for each. They should 
be able to interpret the meaning of a word as it is used in the text. 

When reading literary texts such as fiction, literary nonfiction, and poetry, 
twelfth-grade students performing at the Basic level should be able to de- 
scribe essential literary elements such as character, narration, setting, and 
theme; provide examples to illustrate how an author uses a story element for 
a specific effect; and provide interpretations of figurative language. 

When reading informational texts such as exposition, argumentation, and 
documents, twelfth-grade students performing at the Basic level should be 
able to identify the organization of a text, make comiections between ideas 
in two different texts, locate relevant information in a document, and pro- 
vide some explanation for why the information is included. 

Twelfth-grade students performing at the Proficienct level should be 
able to locate and integrate information using sophisticated analyses of 
the meaning and form of the text. These students should be able to pro- 
vide specific text support for inferences, interpretative statements, and 
comparisons within and across texts. 

When reading literary texts such as fiction, literary nonfiction, and poetry, 
twelfth-grade students performing at the Proficient level should be able to 
explain a theme and integrate information from across a text to describe or 
explain character motivations, actions, thoughts, or feelings. They should be 
able to provide a description of settings, events, or character and connect the 
description to the larger theme of a text. Students performing at this level 
should be able to make and compare generalizations about different charac- 
ters’ perspectives within and across texts. 

When reading informational texts including exposition, argumentation, and 
documents, twelfth-grade students performing at the Proficient level should 
be able to integrate and interpret texts to provide main ideas with general 
support from the text. They should be able to evaluate texts by forming 
judgments about an author’s perspective, about the relative strength of 
claims, and about the effectiveness of organizational elements or structures. 
Students performing at this level should be able to understand an author’s 
intent and evaluate the effectiveness of arguments within and across texts. 
They should also be able to comprehend detailed documents to locate rele- 
vant information needed for specified purposes. 
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Advanced 

( 346 ) 



Twelth-grade students performing at the Advanced level should be able 
to analyze both the meaning and the form of the text and provide com- 
plete, explicit, and precise text support for their analyses with specific 
examples. They should be able to read across multiple texts for a variety 
of purposes, analyzing and evaluating them individually and as a set. 

When reading literary texts such as fiction, poetry, and literary nonfiction, 
twelfth-grade students performing at the Advanced level should be able to 
analyze and evaluate how an author uses literary devices, such as sarcasm or 
irony, to enhance and convey meaning. They should be able to determine 
themes and explain thematic connections across texts. 

When reading informational texts, twelfth-grade students performing at the 
Advanced level should be able to recognize, use, and evaluate expository 
and argument text structures and the organization of documents. They 
should be able to critique and evaluate arguments and counterarguments 
within and between texts, and substantiate analyses with full and precise 
evidence from the text. They should be able to identify and integrate essen- 
tial information within and across documents. 
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Appendix C 



Special Studies: NAEP Reading Framework 

Three special studies were proposed as part of the development of the 2009 NAEP 
Reading Framework. Although very different in topic, they have the common goals of 
improving the quality of the NAEP assessment and gaining maximum information about 
student achievement in reading. One of the special studies (meaning vocabulary) can 
inform test development by providing infonnation about new item types if conducted 
prior to the administration of the 2009 assessment. Other studies propose using data 
gained from the assessment to examine English learners’ reading achievement as well as 
factors that have an impact on the gender gap in reading. Further details about the special 
studies, including methodology, appear in the 2009 specifications document. The special 
studies are presented here in priority order from highest to lowest. 

Developmental Study: Meaning Vocabulary Assessment 



Purpose 

Looking toward the addition of meaning vocabulary items to the NAEP Reading Assess- 
ment, this developmental study will evaluate the reliability and the construct, content, 
criterion, and concurrent validity of the proposed method of measuring meaning vocabu- 
lary. The study was to be conducted well in advance of the 2009 administration of the 
NAEP Reading Assessment to infonn the development and use of meaning vocabulary 
items on NAEP. 

Rationale 

Although NAEP has included a few vocabulary test items in the context of passages on 
past assessments, the number of items was scant and there were no specific vocabulary 
criteria for selecting the items or distractors. Furthennore, past reports from NAEP 
provided little infonnation on how students performed on the vocabulary items and 
whether that performance was associated with comprehension achievement levels; thus, 
these reports did not provide a foundation for emphasizing the importance of vocabulary 
to reading comprehension. The importance of vocabulary in reading comprehension, as 
supported by research, will be much more widely understood and disseminated with 
NAEP’s initiative specifying vocabulary as a major component of reading comprehen- 
sion; NAEP reports providing quantitative data about the perfonnance of 4th, 8th, and 
12th grade students on meaning vocabulary questions and the developmental differences 
among grades; and NAEP reports describing the differences between Advanced, Profi- 
cient, Basic, and below Basic readers on vocabulary and the implications of these differ- 
ences. 
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Recognizing a growing body of research that supports the argument that vocabulary is 
crucial to reading comprehension, the NAEP Reading Assessment includes a measure of 
vocabulary. All vocabulary items are to function both as a measure of comprehension of 
the passage in which the word is included and as a test of readers’ specific knowledge of 
the word’s meaning as intended by the passage author. NAEP will include a sufficient 
number of items to provide reliable and valid data for analysis and interpretation. A de- 
scription of the criteria for word selection and number of items appears in chapter two of 
the NAEP Reading Framework and will be elaborated in the specifications document. 

Research Questions 

What is the correlation between reading comprehension and meaning vocabulary items, 
and how does the addition of meaning vocabulary items affect overall scores on the 
NAEP Reading Assessment? 

How does the introduction of meaning vocabulary items affect the scores of ethnically, 
socioeconomically, and geographically varying groups and low-, average-, and high- 
performing readers? 

What is the correlation between scores on the meaning vocabulary items and a vocabu- 
lary test such as the Peabody Picture Vocabulary Test, 3rd Edition (PPVT-III)? Answers 
to this question will address the concurrent validity of NAEP’s vocabulary measure. 

Special Study: English Language Learners 



Purpose 

This special study will examine the patterns of achievement among English language 
learner (ELL) students and the link between NAEP scores and other indicators of 
students’ ability and achievement, as well as the effects of the accommodations afforded 
students in these groups. 

Rationale 

In today’s schools, the number of ELL students is on the rise. This population trend has 
implications for reading instruction and assessment as educators seek better ways to teach 
and evaluate. Clearly, they need more information about language and its relationship to 
reading comprehension and meaning vocabulary, a link indicated by past studies. 

Although past NAEP reports have provided scores by ethnicity, they have not provided 
information about the link between language minority students and reading ability. This 
special study seeks to examine this link, informing the discussion of how to develop a 
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dynamic assessment (adaptive testing) that more accurately maps the achievement of 
U.S. students. 7 

Research Questions 

What miscues occur most frequently among different ELL groups, and are these miscues 
consistent with different groups of English learners’ speech? 

Are tests of English language proficiency predictive of NAEP comprehension and voca- 
bulary scores? 

What are the differential effects of English proficiency level on NAEP reading and voca- 
bulary? 

How are reclassified fluent English proficient students (RFEP) achieving in comparison 
to other groups in reading comprehension and vocabulary, and how do they progress after 
1, 2, or 3 years of reclassification? 

At what minimum level of English proficiency is a student able to take an NAEP Reading 
Assessment written in English? 

Do accommodations given to ELL students give access to or change the construct of the 
test? 

This study was not conducted because of lack of available funding. 

Special Study: Gender Differences 



Purpose 

This special study examines the differences in reading achievement between boys and 
girls, focusing on factors associated with the gender gap in reading. 

Rationale 

The gender gap — a significant difference between the performance or achievement of 
boys versus girls — exists in a number of education-related settings and situations. Girls 
generally have higher secondary school graduation rates, college admission rates, and 
enrollment in Advanced Placement courses in the humanities, whereas boys have a high- 
er incidence of diagnosed reading disorders. Although boys generally have higher rna- 



7 The ELL special study may be informed by the results of the National Literacy Panel’s study on language 
minority children and youth. The NLP conducted a comprehensive review of research on the development 
of literacy among language minority children and youth that was completed in 2004. 
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thematics and science achievement, the gender gap in the language arts favors girls. Re- 
sults from the 2002 NAEP Reading Assessment indicate the following: 



• The score gap between male and female grade 4 students in 2002 was smaller 
than in 2000, but it was not found to be significantly different from that in 1992. 

• The score gap between boys and girls at grade 8 was smaller in 2002 than in all 
prior assessment years. 

• The score gap between grade 12 boys and girls in 2002 was greater than it was in 
1992. 

• Girls outperformed boys at all three grades in 2002. 

As educators continue to grapple with the gender gap’s implications for instruction and 
assessment, this special study will examine variables in NAEP’s assessment design and 
their relationship to the gender gap in reading. This study will look specifically at the 
NAEP assessment design and at achievement data gathered from the 2009 administration. 

Research Questions 

How are question response modes (e.g., multiple choice, constructed response) related to 
reading achievement? 

How are the types of texts (e.g., narrative, information, poetry) related to reading 
achievement? 

How is the content of the selection (e.g., gender of main character, different themes, 
presence of moral) related to reading achievement? 

This study was not conducted because of lack of available funding. 
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